This article examines the limitations to testing efforts that testers unconsciously apply. Limitations such as premature conclusions, assumptions, biases and industry norms can all cut the thinking process short. Suggestions will be offered on how to identify when you are limiting your testing and how to challenge yourself to keep your mind actively thinking.
Testers actively seek out the limitations to their testing efforts. They evaluate the risks, constraints and dependencies across the scope, schedule and budget; then actively plan and strategize to mitigate those limitations to test meaningfully. When hands on with the system they incorporate diverse testing approaches and consider the complexities and risks of the system under test to gather as much quality related information as possible. Despite well intentioned efforts, many testers unconsciously limit their testing efforts. By being aware of the various traps testers can fall into we can start to understand how these traps can be avoided.
The Industry Norms Trap
Norm's are the rules that a group uses for appropriate and inappropriate values, beliefs, attitudes and behaviors. These rules may be explicit or implicit. The software testing industry has many norms that testers may simply accept, limiting the value of testing and the role of the tester. Testers need to challenge themselves to look beyond what they have learned and generally accepted to be their role and the role of testing. Be willing to step outside the box and focus on how the increase the value of testing and embrace the successes and failures that will come with doing so.
The Personal Bias Trap
Bias is a term used to describe a tendency or preference towards a particular perspective, ideology or result, when the tendency interferes with the ability to be impartial, unprejudiced, or objective.1 Bias can be unconscious or conscious in awareness. Labelling someone as biased in some regard implies they need a greater or more flexible perspective in that area, or that they need to consider more deeply the context. Testers can be biased toward particular methodologies, strategies and tools. Their bias can limit the effectiveness of their testing efforts by preventing them from considering the project context and identifying the "best fit" approach. Broaden your perspective by actively following the industry trends and experimenting with new ideas and techniques.
The Best Practice Trap
Management pressure can often influence a tester's desire to approach their testing efforts with the industry "best practices" to get the job done. The challenge here is that the best practice at Company A working on Project X may or may not have any relevance to what would work best for your current project. Keep abreast of industry trends, however, don't limit and potentially destruct the testing effort by selecting and forcing the implementation of a perceived best practice that might cost your project significant amounts of money and time while reaping limited or worse non-existent benefits.
The Misguided Conclusions Trap
With the onslaught of information in any given day it is easy to conceive how testers may be inclined to take at least some of that information as first hand and infallible. Perhaps the information just seems sound and logical. Conversely perhaps the individual providing the information is in a position of perceived or real authority and/or expertise. The challenge in these situations it that the tester may fall into the trap of assuming the information does not require any further vetting. Testers need to be mindful to always consider the potential that the information is incomplete and would benefit from a line of professional questioning.
The Assumptions Trap
Assumptions are a dangerous thing. We limit our testing efforts every time we stop our thinking process based on our interpretation of the information. This danger sneaks up and catches us even when we believe we are on guard for it. Be wary the next time you think you have “got it” in discussions, meetings, document review, and definitely when testing. Consider if you fully understand the context and if the depth of your understanding is sufficient for the testing effort. Conversely be cautions that you could expend hours of time and energy vetting your interpretation of something with little return on your investment. Realizing that it is not practical to exhaust your understanding, the goal is to be more conscious of when you are choosing to stop thinking about something and move along. Consider the risks of going with your "assumed understanding".
The Déjà Vu Trap
Many times on projects testers get a real sense of déjà vu. They are lulled into a false sense of security that they know exactly how they need to think, act and communicate based on past experiences for this type of situation. The problem is that rarely are products, business domains, customers, etc so similar that the test effort wouldn't benefit from some fresh thinking and possibly a new approach. Testers need to challenge themselves to understand the context and uniqueness of this project and to tailor the testing effort accordingly.
Avoiding the Traps
As humans, there is no way for testers to fully prevent themselves from unconsciously limiting their testing efforts. However, by being aware of these potential traps testers may be able identify their own tendencies. Being mindful there are definitely appropriate situations to limit the thinking and a clear indication of when to just move on. The key is to consciously make the decision, then recognize and acknowledge any risks or limitations that could be placed on the testing effort.
These and other unconscious traps can be the greatest risks to your testing effort. In fact, it is not unreasonable to consider adding yourself to the Risks section of your next test strategy. Having a keen awareness of unconscious limitations is the first step in mitigating the risk of your own thinking process.
1 Wikipedia, http://en.wikipedia.org/wiki/Bias
