The Simple Rules of Software Testing

[article]
Summary:
Simple rules are great for guiding us through an overwhelming workload. Sometimes complicated solutions are necessary, but simple rules often outperform complex algorithms, making them more efficient than sophisticated, difficult flows. They can also break down big goals into practical daily guidelines testers can follow to perform more effectively. Let’s see how simple rules can be applied in software testing.

Simple rules are present everywhere. Of course, some complicated models require equally complicated solutions, like navigating airplanes or performing difficult surgeries. But simple rules often outperform complex algorithms and solutions in numerous areas of life, from rock songs to business strategies.

Simple rules help guide us through an overwhelming workload, which is typical for software testers. The resulting loss is also often much lower in the long run, making the use of simple rules more efficient than sophisticated, difficult flows. They can break down theoretical BHAGs into practical daily guidelines testers can follow to perform much more effectively.

Let’s see how simple rules can be applied in software testing.

What Does It Take for a Rule to Be Simple?

Although there is no strict definition for simple rules, they have some common characteristics.

They have to be short and easy to memorize. Think about training a new colleague: He will have a much greater learning curve with concise, precise rules as opposed to complex guidelines. Short rules also can be remembered and applied more effectively in stressful situations, like production releases or downtimes.

Simple rules also have to be specific to your situation and involve implementing practical activity. The strategy “Customers first” doesn’t qualify as a simple rule because it’s vague and cannot give an exact and practical guide to you daily work.

And the number of rules matters! Simple rules work best when there are only a handful of them. For example, alternative rock band The White Stripes created their widely acclaimed breakout album White Blood Cells with only a few rules: No blues, no guitar solos, no slide guitar, no covers, and no bass. Working within these rules, the band was able to set their creativity free and use studio time effectively. In only ten days, they put together an album that is now featured on many lists of the best albums of the 2000s.

Categories of Simple Rules

Let’s bring it back to software testing. Although simple rules can be diverse on a large scale, they tend to fall into six broad categories.

“How-to” rules: These rules outline key features for how a process is executed. For example: How do you set the state of an item on the kanban board? (For this one, I suggest the simplest setup: Make sure you can set it from every state to every state. You’ll thank me later.)

Boundary rules: If you have two mutually exclusive opportunities, which one do you pursue and which one is outside the pale? For example: What functions are in scope and out of scope in a testing cycle? Furthermore, what level of severity indicates an instant fix, and what severity pushes the issue into the backlog only?

Prioritizing rules: How do you rank the accepted opportunities? These rules can help prioritize a group of alternatives racing for the same resources. For example: What should you test first? A rule can be: Test every issue coming from production first, as they stop generating the direct revenue. But once that’s established, which bugs have to be fixed first? (Priority helps with this, but there are differences between bugs with the same priority.)

Exit or stopping rules: When should you stop an activity? Consider Loeb’s investing rule: If an investment loses 10 percent of its original value, sell it. When it comes to an example from testing, you may need a rule for knowing when to stop testing in a particular testing cycle. (For this, hear my advice in my conference talk “The Magic Number,” where I connect stopping to a pre-identified number of bugs, after which the chance of finding another one is reduced dramatically.)

Timing rules: These rules help synchronize emerging opportunities. Think of this like how Pixar releases a movie every Christmas, so everyone (writers, animators, production, etc.) knows the expected schedule. In testing, you may want to create these rules for your Scrum ceremonies and intervals, or to dictate when to start a particular testing cycle.

Coordination rules: These rules help synchronize the activities of individuals working together. Consider Napoleon’s rule for his soldiers in battle: March toward the gunfire! In testing, synchronizing tools like kanban boards can help you schedule when to test a particular item. This is when the well-known “definition of done” can be used as a test start condition.

Coming Up with Simple Rules

There are some general methods testers can follow for both creating and improving simple rules for their team’s own use.

First, identify a bottleneck that is both specific and strategic. This analysis can be done with an error-guessing technique, root cause analysis, or a shift-left method, among other ways. 

Then, let data trump opinion. Use collected data to decide how to resolve your problems.

There are also several areas you can craft your simple rules from:

  • Your own experiences: Draw from your experience and knowledge
  • Analogies: Use models and similar situations from the past and present (you have to avoid false analogies, though)
  • Scientific results: The greatest benefit is the constant terminology and values, which can serve as common ground between stakeholders
  • Negotiations: Workflows are typical when you have to work with multiple parties, such as negotiating kanban rules with developers, the DevOps team, stakeholders, etc.
  • Advice from experts: Senior software testers can give some great tips for rules to adopt, but be aware that many professionals don’t use simple rules consciously, so you may have to prompt them with questions like, “What did you do in these situations?” or “What’s the reason behind this workflow?”

Simple rules are best when you can also measure how effective your rules are. Use lots of data, but aim for short periods rather than long intervals so you don’t lose your momentum. You can use an issue-tracking tool for this analysis, setting up your queries with the appropriate labels, time, and module conditions. The shift-left method can also be used here, as it can show how to change the testing process based on the previous results.

What are your simple rules for testing? Let me know in the comments below.

About the author

CMCrossroads is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.