Lots of things in the software field have been labeled as "breakthroughs." Invariably, they are not! Inspections, on the other hand, have never been labeled as breakthroughs. But the reality is, they nearly are! Here are some surprising thoughts on what inspections are, and how to perform them effectively.
Probably the biggest plague on the house of software is hype. All too many new techniques and technologies are hyped by their advocates as being the most important new approaches to hit the profession. It happened with 4GLs ("Programming Without Programmers"), CASE tools, object-orientation, Java... and the list goes on. Name your recent software concept and someone probably hyped it. It makes you wonder whether any new ideas are worth anything at all.
Meanwhile, quietly, off in the wings, there is one technology that really is worth some of those hyped claims, but no one is making any exaggerated claims for it at all! "What's that technology?" you're probably asking. It's inspections. Yes, inspections. That old-fashioned technology that we've known about for more than thirty years!
What's so great about inspections? Research study after research study shows that inspections are a better, faster, cheaper way of finding errors in a software product than any of the alternatives. Better than testing. Better than fault-tolerant code.
That doesn't mean that, if you do inspections, you don't need to use any other error-removal approaches. Testing still has a very important role to play, as enterprises that have tried to eliminate it in favor of alternatives have learned the hard way.
Now, what do those research studies show us? That by using inspections properly, more than 90 percent of the errors can be removed from a software product before its first test case is run. That's pretty impressive stuff, and though I doubt the researchers can ever be sure that 90 percent of software errors have been found (that implies they know how many errors are present in total, which is unlikely!), it's clear that more errors were found-more cheaply-by inspections than by any other technique.
At one company having problems with multitudes of undetected software errors, I recommended (as you might expect) that they use inspections. When I followed up some months later, I found that they had, indeed, begun using inspections, and that the incidence of errors was way down. But I was not quite prepared for what the cause of that change had been. It wasn't that the inspections were identifying tons of software errors. But the programmers, knowing that their code was going to be inspected, were poring over it in advance of the inspections and finding most of the errors beforehand.
Fine. If inspections cause a reduction in the number of errors, who am I to care whether that happens at the inspection or beforehand?
So how do you conduct an effective inspection? If you're guessing that I'm going to advocate the well-known Fagan approach to inspections…guess again!
The Fagan inspection, as you may know, is a process-focused inspection. Inspectors are assigned roles and are trained in how to play those roles in the inspection meeting. Meeting goals and processes are rigorously defined. There are meeting rules, and the rules are enforced.
I believe that the product rather than the process should matter most in inspections. And that naturally raises the question: Is there a kind of inspection that is product focused?
Product, Not Process
The answer is "yes." For example, Rifkin and Deimel-several years ago in an SEI-funded study-came up with a way of conducting inspections where the pre-meeting training was in code-reading approaches, rather than meeting process. They tested the two approaches against each other and found that there were "spectacular" benefits to the new approach. Code-focused inspectors were finding 90 percent more errors than process-focused inspectors.
Porter and Votta came up with another approach (they presented it at the 1994 meeting of the NASA-Goddard Software Engineering Workshop [SEW]), which they called "scenario-based" inspection. With this approach, each inspector looked for certain kinds of errors. Once again, the scenario inspectors outperformed the Fagan/process inspectors.
The bottom line of this issue is that inspections that focus on the product clearly outperform those that focus on inspection process. Fagan is by no means the final word on inspections.
To Meet, or Not to Meet?
Is it really important to conduct inspections in a meeting at all? How effective would it be, for example, to turn a bunch of skilled inspectors loose individually on the artifact to be examined?
There are, once again, a number of research studies addressing this question. Bruce C. Hungerford reported at the 1997 Association for Information Systems conference that inspection meetings tend to slow project progress by an average of two weeks, and that the meetings found no more errors than the most competent participant did (although Hungerford admitted he was concerned with the accuracy of that particular finding).
Other studies have found up to an 8 percent improvement benefit in meetings over individual inspections.
What's the bottom line? The jury is still out, but it is safe to conclude at this point that inspection meetings are little better than individual inspections.
How Many Inspectors?
There's another interesting issue here. How many inspectors does it take to do an inspection optimally? There have been lots of research studies of this issue, perhaps surprisingly, and there are several different conclusions. But the answers tend to a fairly clear trend. It doesn't take very many inspectors to begin to approach the optimum point for error detection. One study suggests two inspectors, for example; another suggests three or more, and yet another found that four was no more effective than two.
Based on all of that, you could safely guess that three inspectors is as good an answer as you are going to find.
Inspections are under utilized in practice. To be done properly (that means with intense rigor), they require exhausting, grubby-detail work. It takes a set of experts who really understand both the proposed software solution and the problem domain to do a good job of inspection. Many say that one inspection can only cover about 100 lines of code (again, note that most of what I am saying is about code inspection) before the participants are mentally exhausted. And that translates, generally, into a one- or perhaps two-hour inspection session. Anything longer is counter-productive.
At that rate, doing a thorough inspection of a significant software product could take many hours-hours that the typical project may not allow. Yet those errors not found through inspections must be found later in laborious testing efforts.
In other words, inspection is not an error-removal breakthrough. It is simply the best of a costly collection of alternatives. If you really want to produce software with a minimal number of errors in the production version, use inspections. You may find it painful in the short term, but the long-term payoff is worth it. I hope this column helps point you to some nice ways to minimize that pain.
