Quality: What a Fuzzy Term

In this column (I pompously say) I am going to define the word "quality."

Don't quit reading now. I know quality, as an abstract topic, is boring. I know definitions are boring. But there's something strangely elusive about the topic of quality. In the novel Zen and the Art of Motorcycle Maintenance, for example, the pursuit of what quality means actually drove the main character crazy! And I think part of the problem is that we've been going about it all wrong. In other words, this is a contrarian column. If you like that kind of thing, read on!

First of all, I want to announce, here and now, that I think most people who write about quality don't know what they're writing about. Or more accurately, they actually do know what they're writing about, it's just not quality. Let me explain.

I frequently see explanations in software literature that equate quality to one of these:
 

• a lack of errors
• user satisfaction
• meeting requirements
• achieving cost and schedule targets

Let me dissect each one of these views. For the record, I think all of them are wrong!

First of all, quality is about far more than software errors, or the lack of them. My favorite definition of quality for the field of software-dating clear back to the early Barry Boehm thirty-something years ago-says that quality is a bunch of "ilities." It's reliability. It's usability. It's understandability. It's modifiability. It's testability. It's portability. It's a whole lot of stuff that sums up what it takes to be able to say a software product is of high quality (different people construct different lists of "ilities," but overall they agree much more than they disagree). Given that, how come quality isn't about just, say, errors? Because the error factor is what reliability is about, and that's just a small subset of the total topic of quality.

Now, how about all that other stuff, like user satisfaction and meeting requirements and hitting cost/schedule targets? Well, there is a relationship among all of those things, all right. But the beauty of that relationship is that it shows quality is distinct from all of them.

Here's that relationship: User Satisfaction = Compliant Product + Good Quality + Delivery Within Budget/Schedule.

What does it take to satisfy me as a user of (name a product!)? That the product meets my needs, that it has good quality (e.g., it doesn't fall apart as soon as I get it), and that I can get it when I need it for a cost I can afford. Is that the same as high quality?

For example, I can be satisfied with a McDonald's hamburger if it is what I expect it to be, has a little nutrition, and comes quickly and cheaply. I don't expect McDonald's to produce a high-quality meal, but I can be satisfied with what it does produce.

For a second example, I can be satisfied with a Jaguar automobile if it is as sleek and beautiful as I expect it to be, has plenty of power, is delivered when I want it, and I choose to pay the extremely high cost. But Jaguars, over the years, have not been known for high quality-they have balky electrical systems, for instance. But other aspects of the Jaguar's quality, such as fit and finish, are superb, and may make up for its shortcomings.

For a third example, I can be satisfied with the SAP mega-package information system if it does all the integrated enterprise transaction processing that I expect it to do, with a high degree of trustworthiness and reliability, even if it does cost an arm and a leg (in both time and dollars) to install. With SAP, in other words, I expect a compliant product with high quality, and I'll sacrifice some cost and schedule to get it.

Notice that in the three examples, quality is demonstrably different from user satisfaction, from meeting requirements, and from cost and schedule considerations. Any definition that equates quality with one or more of those topics has simply muddied the waters.

Now, why do I get so excited about all of this? Because quality, for all its fuzziness, is an important topic in the software field. We spent the 1980s, I would assert, trying to find ways of improving the productivity of software people. We spent the 1990s, I would also assert, trying to find ways of improving the quality of software products. That's a lot of years to spend on productivity and quality. I think we all have a kind of intuitive notion about what productivity means; I see no mass disagreement there. But quality? If we can't agree on what quality is, how do we know whether we ever achieved anything at all in its pursuit?

My personal belief is that those major thrusts of the 1980s and 1990s both failed. The thrust toward productivity failed, I think, because we were extremely naive about how easy it would be to make software people more productive. We saw panaceas in things like 4GLs, CASE tools, object-orientation ... (name your own overly hyped productivity poison), and none of them bore the fruit that their optimistic advocates promised.

And the thrust toward quality failed, I think, because of a similar naiveté. But it also failed, big time, because we in the software field never really got around to agreeing on what quality really meant.

So what's my bottom line here? Software quality, at heart, is a bunch of "ilities." Different people construct different lists of what those "ilities" are, but they all pretty much agree on the same basic notions.

Having said what quality is, it is also important to do away with erroneous notions of things that it is not. The next time you read about someone saying that quality is about lack of errors, or user satisfaction, or meeting requirements, or achieving cost and schedule targets, remember this: Quality is none of those things. But it has an important relationship with all of them.

Editor's Note: Robert Glass has written a follow-up to this column in response to the many comments: "Revisiting the Definition of Software Quality."