A software tester can begin testing early—very early—before the software has even been built. Karen Johnson explains that one of the best times to start testing a product is in the product-discovery phase.
A software tester can begin testing early—very early—before the software has even been built. One of the best times to start testing a product is in the product-discovery phase. From my own experience and perspective, I feel strongly that I participate and contribute from product ideation to delivery and beyond. On numerous occasions testers have asked me how a tester can help during the product discovery process. I want to share how a tester can do so from the beginning of the early product idea sessions all the way through development and even ahead to the next sprint.
Product Discovery Sessions
When team members work together during product discovery sessions, they are thinking through ideas for how to build a product to meet customers’ needs. The team’s focus is on designing a usable, deliverable product. During the product sessions, a tester can actively listen to the product being designed and think through user scenarios and integration points with other features and possibly integration points with other applications. By thinking with a testing perspective early on and sharing these ideas, in addition to asking questions early in the process, a tester helps the team uncover ideas for how to build the product better and maybe even how to build a better product. Sometimes the team’s tester knows the integration points better than anyone else on the team. Because of our knowledge of existing defects, we are in a position to help avoid introducing similar issues. Just one question posed early in the process can impact how something is built. As I see it, preventing a bug from being built is even better than finding a bug.
Build a Mental Model and Then Break It Down
Envision the software already built. Now ask yourself how you plan to test the software. What areas or aspects of your mental model are unclear? Is there any aspect of your mental model that you cannot envision or seems unclear in your mind? This may include problem areas that have not been well thought out by the team. Remember to ask questions, because your questions can help the team see the parts of the design that need more thought and more planning. By asking questions, you can finish product discovery sessions, be ready to plan the testing efforts, and, more importantly, contribute to a better-designed product.
Once you have your head wrapped around a mental model of what is being built, ask yourself the following questions: How can I destroy this model? What weaknesses come to mind? What are the vulnerabilities? Have I tested anything like this before, and if so, what defects were there? What “what if” scenarios can I think of? Can I now share these testing thoughts with the team?
Defects found earlier in the development cycle are the easiest and the cheapest to fix; imagine finding a defect before the software is even built!
Being able to break something that you cannot see, software that does not exist yet forces you (in a positive way) not to rely on the user interface or being able to actually use the software in order to find defects. Instead you’re almost entirely forced to think of the big picture instead of the specific screens of the software or pages of a website, instead your thinking is centered on the purpose of the software without the nuances of navigating screens to find flaws in the flow or the process of the software. The more you’re able to envision software before it is built, the more you can help in product-discovery sessions.
Working with an MVP
When a new feature is being built, agile teams often focus on building an MVP—the minimum viable product. This means that the team is keen to build the most basic version of a feature first and to get the feature to market in order to receive customer feedback before the feature is built out in full detail. When team members are focused on building the most basic version of a feature, they may be frustrated dealing with the lone tester on a team. This lone tester thinks and plans for testing, imagining a battery of tests for something that is in its most basic primitive “draft” stage. Your testing ideas and strategies can be refined at the same time the software is being refined; early testing ideas should be rough just like the first versions of the software. Sure, you need to test the software, but when a sketch of a feature is just being built, your testing approach needs to mirror the stage of completion of the developers and the code and not be stages ahead.
Pairing Along the Path
If a developer is willing to share an early build, ask what’s working and what’s not. Don’t report defects in areas that haven’t been coded yet. You might mention the issues you found but do so in a way that is thoughtful, so that you’re helping the developer and not pointing a figure at “what’s not done yet.” Time and again I have found developers become unwilling to share early code with testers who test ahead.
As you continue to develop the product, gradually increase the rigor of testing by trying more complicated scenarios and more challenging test conditions. As the software is “firming up,” this is a good time to move to security testing and negative testing; think less about confirming testing and more about breaking it.
When a developer shares what unit-level tests he has built or if you’re able to review the actual code and unit tests, what additional testing can you execute to both supplement as well as complement the unit testing? Now is the opportune time to think of other scenarios to test, think of integration or end-to-end testing paths you can now investigate. Is the code strong enough to be load tested? If yes, plan with the developer who and how to execute load testing. As soon as the software is developed enough, you can begin security testing as well. Ramp up the testing in coordination with your developers and plan your pacing in collaboration.
Winding Down the Sprint
As the sprint winds down and the MVP gets ready to be delivered, think about conducting automation (if not sooner). By the next sprint, there will be yet more defects for retest, more new features to test, and what was once new during this sprint will become technical debt. Now that the MVP of a feature is built, consider building some test automation if you have not already. Now that the feature is fresh in your mind, the details clarified, see if you can record notes or ideas for future testing that can be streamlined for the next sprint. Use this time to build data sets for reuse. Make sure all the defects you have discovered have been logged and read clearly and completely.
A final sanity check may be needed to reminisce the product discovery session – has the team built what was discussed? Think back to the original need that was discussed, does the product that was built reflect the needs discussed at the start of the process? Perhaps the team’s thinking and understanding of a feature changed shape from the time of the first sessions to product delivery – did your testing ideas evolve as well? Instead of being the gatekeeper at the end of the process during waterfall development, you’re now a fully integrated team member in the agile development process in which each team member is responsible for what gets delivered. From my experience, it is a fantastic way to feel part of the team; there is not only collaboration, but also camaraderie.
For Testers Working Offshore
Testers need to show up at the sessions! For testers offshore or not onsite, it takes extra coordination on their part to join via phone or web conferencing. From my work experience, I often encourage testers to speak up, be part of the team from the start, and not be shy. Be focused on contributing.
For Testers Shifting from Waterfall to Agile
For testers whose background is waterfall-based development, asking questions early on may seem unnatural. In the past, testers would read requirement documents and plan testing from documents. For an agile team, these early product thoughts are much more likely to be a conversation. Because of this, testers shouldn’t be shy or afraid to chime in with their ideas. For testers who are not used to questioning the team but who are used to accepting requirements documents, “as is” the process of questioning how a product will be built versus questioning what has already been built may require a shift in thinking. In an agile context, testers are in a better position to not only question whether a feature was built right but also whether the right feature was built.
