Start a Revolution

I don't think that it's a tester's job to find bugs.

There, I said it. I have been wanting to say it—even shout it—for a long time, but I was finally compelled to do it by Lee Copeland's column "Institutionalizing Poor Quality" and the many comments it received. Remember the example he gave where a QA person pops up to inspect the pastrami you are about to buy? He offered this example, among others, to point out that we routinely expect a certain level of quality from most professions, so why don't we expect it from programmers?

Let me take it one step further. Let's say that you are forced to wait on your pastrami while it is inspected—and you are getting hungrier by the minute—then once it is over, your purchase receipt reflects a 20 percent surcharge for the inspection. It might prompt you to question the value of this service. After all, having it too thick or too thin might not matter that much to you. The reason it is worth it, you are told, is that the inspectors are also looking for E. coli bacteria, which could make you seriously ill or even kill you.

Okay, now quality has your attention. But, as you think about it, you would likely wonder why the E. coli inspection doesn't happen much earlier—say, when the meat is slaughtered or packaged. Or, maybe the butcher's premises should be inspected routinely. Wouldn't that be much more time and cost effective than inspecting every single slice? Of course it would.

All of this leads back to the difference between quality assurance and quality control. I used to think people who emphasized this distinction were being anal retentive, but I have come to see their point. Quality assurance is about building quality into the process, and quality control is about inspecting the end result.

As Lee concluded, "In our world, the process that needs to be improved is the development process."

But if you are a tester, what does this mean to you?

First, let's face the facts. Testers did not ask for the product, did not design it, did not build it, will not pay for it, and will not use it. So it's fair to ask—what does testing bring to the party? If the answer is "to find bugs," then I think you are doomed to being overworked, underpaid, and under-appreciated, if not downright vilified. Why? Well, for the simple reason that you can't prove a negative.

Think about it. What you are saying, essentially, is that there are an indeterminate number of bugs in unknown locations of unpredictable severity, and at the end of the project all you can say is which ones you have found but not how many are left. Is it any wonder it's hard to put a high value on this service?

If I were to play the role of the customer, who is having the software developed at great expense to meet a critical business need, here is what I hear you saying: "I want you to pay me to delay your use and enjoyment of this critical software so I can see how many things I can find wrong with it." Not very appealing, is it?

Don't get me wrong—there are most likely bugs out there, and some of them might be bad, but the problem is that we just don't know, and it's hard to get someone to pay for something that can't be measured.So what's the alternative?

It's the requirements, of course. Customers don't have software developed so that it will be bug-free, they do it so that it will meet a business need. In fact, companies use software every day (and so do you) that has bugs in it that are known and—in some cases—have been for years. As one customer told me recently, "We have a bug that has been in our system for over ten years and everyone knows about it, but every time we prioritize requests for enhancements and fixes, this bug is at the bottom of the list."

So instead of making it your mission to find bugs, make it your mission to verify the requirements. Now you might say, and rightfully so, that the requirements are either missing, obsolete, or incomplete. But so what? Make it your job to find them out, because that is what the customer is really paying for.

And you might also say, and experience will back you up, that there may be really nasty bugs out there that are on the fringes or even outside the expected pathways, and you will never find them just testing requirements. Again, so what? You can't guarantee that you'll find them even if they are there, and if you do find them you can't be sure that the customer will even care about them so long as they can be worked around or avoided.

Back to role-playing. Now, here is what I hear you saying: "I want you to pay me to assure that this major investment will meet your business needs without jeopardizing critical operations." Now you're talking.