Quality Metrics for Test: Evaluating Products, Evaluating Ourselves

As testers, we usually focus our efforts on measuring the quality of products. We count defects and organize them by severity, we compute defect density, we examine the changes in those metrics over time for trends, and we chart customer satisfaction. While these are important, we must apply additional measurements to ourselves if we are to reach the next level of testing maturity. Lee Copeland suggests that we (1) count the number of defects in our test cases and the time to find and fix them; (2) compute test coverage, a measure of how much of the software we have exercised under test conditions; and (3) determine Defect Removal Effectiveness, the ratio of the number of defects we find divided by the total number we should have found. Start keeping these and other metrics, and we are on the way to improving our testing processes and results.

• Four basic measurement types-counts, counts over time, ratios, and surveys
• How these measurements can be applied to product quality
• Measurements to evaluate the effectiveness and efficiency of the testing process