When does a groove become a rut? Elisabeth Hendrickson has experienced both grooves and ruts while testing a seemingly endless software project or series of projects. Here she explores the subtle slippery slope from groove to rut. She also discusses common causes. Once you fall in, Elisabeth has some pointers to help you climb out.
Groove. Groovy. In the groove. The word "groove" has positive connotations. We're in the groove when we're focused and productive. When I'm finding lots of bugs in new software and simultaneously learning a lot about the system, I'm in a groove. However, being "in a rut" has negative connotations. When I'm doing the same thing over and over again, I'm no longer in the groove, I'm in a rut. So how do you know when a groove becomes a rut, and what can you do about it? It's a subtle change that slowly sneaks up on you. Comparing activities day to day reveals little difference. Yet the more we wear the groove, the deeper it gets.
Imagine you're a tester assigned to find bugs in a project. With the first build of the system into test, you're just getting started. You're not yet in the groove. By the next build into test, you've got a handle on things. You're beginning to get an idea of where the weak spots are and you're having fun exploiting them to find serious issues. A few more days into the project, somewhere around build 3, you become a bug-finding machine.
But by build 10 or 11, you're beginning to dread new builds coming into test because it means rerunning tests to ensure nothing else broke as a result of recent changes. Your energy flags. You begin to wonder if the developers put the bugs there on purpose just to test your bug-finding skills. Within a few more builds you're spending most of your time running tests you've already run. You're using the same test data as before because it would take too much time and energy to come up with new test data.
Now you're in a rut. The longer the project goes on, the more deeply entrenched you become. If subsequent releases of the software have only minor changes, you may even extend your rut from one project to the next. I know because I've done just that, only to wake up one day and realize that I haven't done anything new for months. Being in a rut is a dangerous state for a tester. Given that there are an infinite number of tests we could conceivably run on any given piece of software, we need as much creativity, originality, and energy as we can muster. Certainly, there are times when we need to rerun tests. However, even when rechecking areas of the software we've already tested, we can vary the tests. Varying tests and test data ensures that we're looking at the same functionality, just in a different way.
Taking the "Groove Test"
When working on a long project, I monitor the depth of my groove to ensure it hasn't become a rut. Whenever I've been working on a project long enough to get in the groove, I begin asking myself these questions at the end of every day:
1. Do I still feel like I'm in a groove?
2. Have I done anything innovative today?
3. How could I update my test data?
4. Can I think of at least three different ways to run each of the tests I just ran?When I notice internal resistance to change, I can tell that my nice little groove is about to turn into a perilous rut. That's when I know I need to make changes. Maybe I can get a snapshot of some different production data to test against. Maybe I can add some complexity to my tests. (After all, by now the software should be mature enough to handle harsher tests.) Perhaps it's time to look at the system from another perspective. Or maybe I can work with another tester to think up some truly devious ways of stressing the system. Now we're talking!
I have a fifth question I also ask myself:
5. Am I still having fun?
When it stops being fun, I know I'm no longer being as observant as I need to be. That's when the nasty ones sneak by me. I fail to notice signs that all is not well. I declare a set of features to have "Passed" when I didn't set any real checkpoints at which the bugs might have been caught.
Failing to spot bugs because we're too deep in a rut can happen to any of us, whether we pre-plan tests or use an exploratory approach. That's why two different testers can take the same set of tests with the same version of the software and find vastly different bugs. The difference in the quality of their efforts is in their level of observation. One of my early managers declared that our test group achieved "Quality through Vigilance." It's hard to be vigilant if you're asleep at the keyboard.
So what do you do if you've just discovered you're in a rut?
- Throw away the test documentation for a day, pretending you're new to the project and the software.
- Analyze the software under test in a different way. If you've already done boundary analysis and flowcharting, try looking at the architecture of the system to see the big picture. If you've looked at the relationship between the pieces and parts, try designing real-world use cases.
- Change your data. If you're using test data, get real-world data. If you're already using a snapshot of production data, get production data from a different system. If you've exhausted all your sources for production data, combine the data you have from varying sources into a gigantic set of diverse test data.
- Change your timing. Time is an oft-forgotten dimension of testing. Try doing the same thing but at a different pace or with a different rhythm.
So take the five-question "Groove Test" to gauge your ability to see new possibilities and act on them. Whenever you start resisting the idea that your tests might need to change, it's time to change them!
