Top 5 Bogus Excuses for Not Using Automated Testing Tools

I have been helping companies test their Web applications for the last six years. In the course of my work, I have run across many developers and QA pros for whom automated testing has been a godsend, enabling them to deploy high quality, critical business applications on time and with confidence. I’ve also run into people who claim to have no use for automated testing (I suspect they've been burned by old-school client/server testing tools, which had had a reputation as money pits, and most of which sat on a shelf gathering dust while shell-shocked users retreated to manual testing methods).

So, with apologies to David Letterman, here are my Top 5 Bogus Excuses for Not Using Automated Testing Tools.

1) I don't have time for formal testing, our schedule is too tight.
This is an oldie but goodie. Has there ever been a development project that went according to schedule? Does anyone know of a project where QA time wasn’t cut at least in half in order to meet the deployment timeline? Automated testing can actually help a great deal in this scenario, enabling you to perform more tests (and more types of tests) in less time. Put another way, you can spend less time scripting, and more time testing using automated testing tools.

2) It's too expensive.
Don't let a few bad apples spoil the whole bunch. Not all automated testing tools are overpriced, and not all vendors are looking to gouge you. Testing tool licenses start at less than $10,000 and hosted load testing solutions can bring down the cost even more while eliminating the need for you to develop a load-testing infrastructure. And don't even get me started about the cost of unplanned application downtime. A leading technology research group estimates that an online retailer will lose nearly $60,000 an hour if a critical application goes down. An online brokerage will lose $537,000 for just five minutes of downtime. Given those figures, doesn't it make sense to fix potential problems before they lead to downtime?

3) They're too hard to use.
I know you've been hurt before. Legacy testing tools, most of which were originally developed for client/server environments, can be a bear to use. Some even require proprietary languages. But a new class of Web-based testing solutions enable you to create very complex scripts with no programming in just minutes. If it's been a while since you evaluated testing tools (i.e., more than two years), it would be worth your while to see what’s out there now.

4) We load tested it manually.
I’ll try to break this to you gently–you can’t load test applications manually unless your expected load is smaller than your development team, and you can duplicate the production environment in your office. Companies have actually said to me, “"We're all set–we all stayed late one night and logged on to the application simultaneously–it worked fine!" Chances are, your application will find its real-world load a little more taxing. Automated load testing is the only way to see how your application will truly hold up under a variety of load scenarios.

5) We were very careful–there aren't any bugs.
This is my favorite. No developer likes to think there could be any problems with his or her application–but the fact is, I have helped hundreds of companies test their applications, and I have NEVER been through a test that didn’t find at least one problem. More often than not, we find several major ones. A colleague has observed a particular psychological phenomenon within development teams that trot out #5 as an excuse. It's modeled after the better known Four Phases of Grief, and he calls it the Four Phases of Software Testing.

The Four Phases of Software Testing are:

• Arrogance–"You're just here to verify that my application is perfect."
• Denial–"You're not testing it right–there's nothing wrong with that feature. Try it again."
• Pleading–"Oh no! Can you help me fix it?"
• Acceptance–"OK, now we know how to avoid that situation next time. What are we testing next?'

By the time they reach Acceptance, most people have converted.