The communication of functional requirements and specifications is the most difficult, critical, and error-prone task in IT projects. Research has shown that projects that proceed to the construction and coding phase with missing or wrong functional requirements and specifications are almost certain to fail. To avoid missing or misunderstanding the requirements for a solution, and to avoid the development of systems to incorrect specifications, we need a systematic approach to capturing, organizing, and validating the functional requirements and specifications. In this article, Bill Walton offers such an approach.
The Challenge
Researchers first documented the sources of errors in software development almost twenty years ago ("Sources for Errors in Systems Development," Basili & Perricone, Communications of the ACM, January 1984). According to that study, slightly over half of the errors (52%) were generated in coding and construction, were caused by earlier fixes, or were of a miscellaneous origin. The trend in the industry has been to focus efforts here. Development tools and methodologies have been developed and commercialized with great fanfare.
Despite this, project failures have not materially decreased. The problem persists because those are not the errors that cause projects to fail. The vast majority of errors made in coding and construction are simple-easy to find and fix. The root of the problem lies elsewhere, in errors that result in wrong or misunderstood requirements or functional specifications. These are the errors that cause entire designs to be revisited and revised. These are the errors that result in negative impacts to other systems or user groups. These are the errors that cause projects to fail.
The definition of functional requirements and specifications is a difficult and error-prone exercise in communication. The primary difficulty that we, as analysts, encounter is maintaining the right context for information as it is delivered. When we have the right context, the information makes sense and we capture it. When we lose the context, it doesn't make sense to us and we may misunderstand the information being communicated. What we need is a systematic way to capture all the information we're given and to organize it so that missing information is more easily identified, and the full impact of change is more easily understood. We need a systematic approach that better facilitates having our understandings and related recommendations validated or corrected.
The difficulty of our task is driven by complexity. Effective communication consists of information transfers that take place within a context that both parties understand. Our communications take place not within a single context, but along two separate axes of context. The first axis contains the procedural, organizational, and technical aspects of the business that may or may not need to change to arrive at a workable solution. Within each of these, we contend with the second axis. Some users talk about the way the system works today: "I don't like the way X works." Others talk about the way they want it to work when the project is complete: "X should work like this." Still others describe the changes they think we should make: "If you do A, B, and C to X, that will fix it." Some will even present us with the solution as they envision it: "Here's what X should look like. It is not unusual for an interviewee, within a single conversation, to deliver a mix of all of these. Given the complexity of the communication, it's not surprising that requirements get missed or misunderstood.
A New Approach
A critical first step in solving the analysis challenge is defining the terms we will use to communicate. The words "problem," "objective," "requirement," and "specification" are particularly important. Using the same words to mean different things, or different words to mean the same thing, is a common source of misunderstandings. The approach I describe here uses the following definitions.
The problem addressed by the project is a description of the way the thing works today. The objective is a description of the way the thing should work in the future, when the project is complete. The requirement is the difference between the way the thing works today and the way it must work in the future. That is, requirements are the changes that must be made. The specification provides a detailed description of the new element that will satisfy the requirement. It may be a mock-up of a new or changed screen, a file layout, a business process diagram, or any concrete, unambiguous representation of the thing being changed that facilitates communication for the purpose of validation. The important thing about these definitions is that they accommodate the recording, without modification or interpretation, of all of the information we will gather.
The key to the approach described here lies in applying this structure (i.e., problem - objective = requirement) consistently; whether "the thing" being described by the interviewee is a business process, an organization structure (e.g., roles and responsibilities), or the IT systems and infrastructure supporting the business. The result is the multidimensional matrix shown in Figure 1. Each piece of system functionality is enumerated, and the question of whether or not it will be changed is explicitly addressed. Each piece of system functionality is linked to a task within a business process and each task is linked to the individual or individuals within the group or groups that perform it. When analysis is complete, all the cells in each layer of the matrix will have contents, and each cell in any layer will be linked to a related cell in each of the other layers. The contents should be consistent within each layer of the matrix but each layer will likely have a different type of content (e.g., business process diagram vs. org chart vs. UML diagram). The matrix is easily implemented using a spreadsheet. The linkage between layers can be as simple as a text reference within a cell that manually leads the reader to the related cell within another layer. At the other end of the spectrum, depending on the tools available and the technical skills of the analyst, a Web-based, hypertext format can also be used to allow a broader audience of readers to navigate the matrix automatically. This basic structure can be extended in both depth and breadth to accommodate additional information-gathering needs.
It is often desirable to utilize a layered, hierarchical approach to documenting each of the contexts: organization, process, and technical. This makes is possible to capture and organize, for example, all of the related IT systems supporting a business function and then "drill down" with separate sheets to each of the impacted systems and their specific functionality. The drill-down can continue to the level of individual fields on a screen that are then tied to sub-steps in a business process that has been similarly decomposed. This can be particularly useful in documenting exception processing when the project addresses a new system that automates a previously manual process. It can also help to materially decrease the probability of unintended impacts to other IT systems.
Figure 1. Requirement Communication Matrix
Another extension, one that proves especially helpful when dealing with new systems that implement new technology, is the addition of the infrastructure aspect of the project. The inclusion of this aspect allows the analyst to include the business processes and organizational roles and responsibilities of the MIS group that will support the new system. Just as each piece of system functionality supports a task within a business process performed by an individual or individuals within one or more groups in the business unit, each piece of system functionality is provided by one or more infrastructure components supported by an individual or individuals within one or more groups in the MIS unit. The infrastructure sheet(s) utilize the same structure as their counterparts on the business side of the documentation and will also be tied to the IT system sheet. Similarly, the structure can be extended to include the help desk aspect of the support function and the performance aspect of system functionality. These extensions can greatly improve the probability of identifying training and staffing requirements that would otherwise go undiscovered until much later in the project.
The Benefits
The key benefits of using this approach center around improved communications to increase the likelihood of identifying missing or misunderstood requirements and unintended impacts to other systems or user groups early in the project. We have to assume that everything a user tells us is important; that it contains information we need in order to succeed. By providing a container capable of holding any information we gather, the approach described here allows us to separate the data-gathering activities (focus on listening) from the analytical activities (putting the information gathered into the correct context). This allows us to focus all our energy during the interviews on active listening. By providing a matrix that encourages the enumeration of all of a system's functionality and the articulation of the past, present, and future of each of those pieces of functionality, the approach makes it easy to identify missing requirements. The first round of interviews with a user community typically ends with much of the needed information missing. Each layer of the matrix will look much like a checkerboard with some the cells filled and others empty. The structure highlights the missing information and allows the analyst to focus future interviews on obtaining the information needed to complete the matrix. The consistent use of explicit definitions that allow the information gathered to be repeated back, without modification or interpretation, within what the analyst believes is the correct context, facilitates the identification of misunderstandings.
By providing a structure that ties business processes to IT systems and organizational roles and responsibilities, the use of the matrix facilitates the early identification of unintended impacts to other user groups or to other systems. Business process diagrams may identify other groups involved in the process but not currently part of the project. Screen shots can identify fields that are not part of the current project, triggering questions about other users of the system. By identifying these things early in the project we increase the probability of success by making it clear that we need to do one of three things:
- Increase the scope of the project to include these systems or groups
- Reduce the scope of the project to exclude functionality that would
impact them - Shelve the project until business constraints allow their inclusion
Any one of these actions allows us to avoid a failure that would otherwise result.
Conclusion
Successful IT projects address all the relevant aspects of the business that need to change to solve a business problem or seize a business opportunity. Defining requirements as they relate to the development of a software application is difficult enough. But it is not enough to guarantee success. Business processes, roles, and responsibilities must also be examined for required changes. This greatly increases the complexity of the communications. Avoiding the failures that have plagued IT projects in the past requires a new approach that facilitates complex communication and the early identification of missing or misunderstood requirements and specifications. The approach described in this article uses a simple structure within which effective problem-solving communication can take place. This structure both tracks and guides communications to a complete and correct result.
