Re: System Measurability LO3375

mbayers@mmm.com
Mon, 23 Oct 1995 09:19:07 -0500

Replying to LO3356

Ben Budiman writes:
>In other words, I am wondering if any of you is aware of a criterion,
>or something similar, that qualifies a measure as a 'true'
>representation of a system of interest.

A team of people at my company has recently undertaken an effort to
rigorously address just what we mean by 'quality' in the information
systems which we build. Management initially gave us the mandate to
improve productivity -- we figured that we must first determine how to
-measure- it before we can try to -improve- it. And that in turn means
implies measuring the 'goodness' of what we build.

During the course of some lively and instructive dialog, we concluded that
-any measure- of the quality of a system seems inherently subjective. We
compared a typical college biology test with a typical college English
test. The first has 150 multiple-choice type questions, while the latter
has three essay type questions, of which you must respond to two. The
biology test somehow seems more objective and the English test more
subjective at first glance.

But on further examination, we concluded that whoever picked out the
biology test questions presumably used a subjective set of criteria -- to
discover how much the student had learned about what the test-builder
-thought- were the most important parts of the course. That is, the test
builder has disguised the subjectivity, but it remains at the foundation
of the -evaluation-. Then we noticed that -value- lies at the center of
evaluation.

So doesn't any measure of quality depend on someone's value scheme? And
doesn't the 'ladder of inference' (5th Disc Fldbk, p245) remind us that
that the value scheme affects what we observe?

Michael A
--
Michael Ayers
mbayers@mmm.com        (612) 733-5690      FAX (612) 737-7718
IT Education Svcs/3M Center 224-2NE-02/PO Box 33224/St Paul MN 55133-3224
- Ideas in this note only represent the author's attempt at thinking and
- certainly do not represent the positions of anyone else in the galaxy.