We create lists to help us prioritize tasks and stay on schedule. Sometimes those lists help us accomplish those tasks faster. Sometimes those lists simply chain us to an archaic way of doing things. Having a "To Do" list is a good thing if you don't let it prevent you from thinking outside the box. In this column, Elisabeth explains why the agenda items that don't make the list can often be some of the most important.
I have a "To Do" list. Actually, I have several. I have Web site update "To Do" lists and "Client To Do" lists and "Writing To Do" lists. But I digress.
Every time I make a commitment, I try to remember to put an item onto the appropriate "To Do" list. My usual habit is to remove items from the list only once they are complete. And that means that my "To Do" list can get very backed up. I often struggle with the feeling that I must complete everything on my "To Do" list, an impossible task. Worse, I sometimes feel required to finish certain onerous "To Do" list items before I move on to the more interesting tasks on my plate. Some of the onerous tasks may be less strategically important than the fun ones, but they're on my "To Do" list, so I feel they must be done. The end result in some cases is that neither the onerous nor the fun tasks get done. I'm stuck.
In the worst cases, I've had to throw out that "To Do" list and start over. This is not ideal—I'm throwing out the good with the bad. But sometimes it's the only way to get myself unstuck. A better approach is to comb through the "To Do" list, plucking out those items that truly still need to be done and then tossing the rest. But by the time I get stuck, I'm incapable of doing even this kind of weeding. The whole list has to go.
When I let myself get stuck on a handful of "To Do" list items, I am suffering from the tyranny of the "To Do" list. I am allowing my life to be run by line items stored in my planner. That just doesn't work. I can't allow my "To Do" list to run the show. Yet I see the same kind of "To Do" list tyranny happen in some test departments. "We have the test in our suite, we have to run it," they say. "No, we can't add more tests. No, we can't improvise. This test is here; it must be run. It leaves no time for any other testing." What a sad situation.
There are an infinite number of tests we can run. Of those, some are more likely to yield interesting information than others. In testing, we're after the interesting information. Unfortunately, some test suites seem specifically designed to not find out anything interesting. The documented tests might have been interesting two years ago, but they yield no new information now. Wouldn't it be a shame to miss a serious bug because we're too busy running the planned tests?
Think it can't happen? I've seen it. For that matter, I've done it. I've seen entire departments slog through the tests cases in their documented test suites because they were "supposed to," all the while bemoaning the fact that they didn't have time to do the interesting testing. In one such organization, one tester argued with her manager for the freedom to run some unplanned tests. "No," said her manager. "If you think it's important enough to run, it's important enough to document. Put it in the test case database, and then you can run it." Mindful of the time these updates were taking, the tester only added those tests she thought were most critical.
As a result she nearly missed a showstopping bug. Fortunately for her project, she found it just in time—because she deviated from her test suite while testing a bug fix. Her manager finally relented. He gave her a little time to do exploratory testing. He wondered if she could she put an item in the test case database called "Exploratory Testing," though. This is the problem with following "To Do" lists."To Do" lists are a guide. When used well they help prioritize tasks. When they're allowed to take over, they become a trap, forcing list followers to do things that are no longer important. Test suites are also a guide. They point us to areas that we must test. Our test suites remind us of all the ways in which we agreed to exercise the software before shipping. We need to pay attention to those tests suites without allowing them to overcome our good judgment.
Just as I'm learning to treat my "To Do" list as a guide rather than a rule, let me suggest that you let go of the documented tests long enough to figure out if they really are the most important. Try unexplored scenarios. Look at the software from a new perspective. Do crazy things you think no sane user would ever try.
Your team could be suffering from the Tyranny of the Test Suite if you're ignoring certain testing tasks in favor of running predefined tests. Veer away from the established path for a time so you can come back and look at your test suite with fresh eyes. Does it truly contain the best and most important tests? Or has it become a petty tyrant forcing you to waste time on unimportant activities?
