Using CM to Enhance the Testing Process

• • A central repository for all QA Test Cases
  • Ensure that only authorized users can modify Test Cases (e.g. Testers, QA Manager)
  • Checkout/checkin control of Test Case documents
  • Versioning of Test Case documents
  • Provide a life cycle, approval methodology, and audit trail for the modification of Test Cases
  • Email notifications when Test Cases are modified
  • Allow for the linking of Test Cases to Requirements
  • Allow for the linking of Test Cases to Bug Reports

It's usually a good idea to define terms, so let's define a Test Case for purposes of this discussion. At my last employer, the test engineers had accumulated an inventory of nearly 500 pre-formatted Word documents that they used to document the test scenario for each business requirement. These documents contained the Test ID, Test Name, Description, Test Set-up, and step-by-step test instructions (i.e. Test Script) for each test scenario. These Test Cases would be executed in their entirety (as a regression test) or in part (as a final functional/acceptance test) prior to distributing a release to production.

Initially, the Test Cases contained the test results as well. The testers would copy the Test Case documents en masse as templates for each release, conduct their tests, and document their results in the release-specific copies of the Test Case documents. As part of the conversion to CM control, we decided to simplify the process by removing the test results from the Test Case documents. The test results were migrated to a separate Test Results Matrix (one matrix for each release). This step allowed us to migrate the Test Cases as a single set of templates into the CM repository.

With an Automated Test Tool (ATT), the Test Script or even the entire Test Case might be stored and managed within the ATT itself. We did not enjoy the luxury of an ATT, so the Test Cases were maintained manually. If your shop has an ATT that stores the Test Scripts internally, then there may still be a need for a Test Case document to serve as a "wrapper" for each Test Script. Otherwise, the entire Test Case may end up in the ATT, and CM would not need to manage the Test Cases at all.

As a Configuration Manager, I worked with the testers to design and implement the Test Case repository and maintenance process. The steps to convert Test Cases to CM control are roughly identical to the steps one would follow to convert a software application to CM control. Assuming your CM tool can manage Word documents just like any other configuration item, the steps are:

• Identify and resolve conversion issues (as noted above)
• Design and build an appropriate CM repository structure (i.e. folders/subfolders)
• Load the Test Case documents into the CM repository
• Implement native OS security rules to restrict external access to the repository
• Implement internal CM security rules to restrict functions by group/role
• Implement process rules, development areas, and other features (e.g. email notifications, approvals) to establish a life cycle for maintenance of the Test Cases
• Provide some quick, informal training to the QA staff.

The process was simple and straightforward. Each tester had a development area on their own workstation, and they would checkout Test Cases as the need arose (e.g. a Requirements change), make their changes, and then checkin the modified Test Cases. Our CM tool had a nice interface to MS Word that allowed the testers to modify the Test Cases without leaving the CM client, and it used the MS Word "Track Changes" feature to highlight differences. After an approval by the QA Manager, the CM tool would automatically complete the checkin process to the CM repository, and the team would be notified by email. Obviously, you would have to modify this design to fit within the features and functions offered by your firm's CM tool suite.

Jim Zitzer is an independent consultant with over 15 years experience in application development and software configuration management. You can reach Mr. Zitzer by email at [email protected]