Linda Hayes is a practitioner and proponent of test automation, but she recognizes that much of testing remains manual. The unexpected twist, though, is that she lays the blame for automation's chilly reception squarely on the shoulders of early approaches to automation—namely capture/playback. In this article, Linda takes a look at automation's history and offers some suggestions for its future.
As someone who has made a career out of test automation, I find it downright embarrassing that the majority of testing is still performed manually. You might think—as I have been tempted to—that this is simply the result of uninformed or apathetic management's refusing to commit the time, money, and resources to make it happen.
But you and I would be wrong, because billions of dollars have been spent on test automation tools, training, and resources. Virtually all companies of any significant size are not only aware they need automation but also desperate to have it.
So, what's the problem? It's all those billions that already have been blown—er, spent. Those who championed the expenditures are not eager to claim failure. And it's not as though all was lost; the test management and reporting capabilities of test suites often survive because even manual tests need to be managed. But automation itself has remained elusive, and the managers who approved the purchases are left wondering why it takes so long and where is their return on investment.
What Went Wrong?
Sad to say, but we automation pioneers have only ourselves to blame. In the early heydays of automation, we convinced customers that capture/playback was a quick and easy answer to automation. Just turn on the recorder, perform the test as always, and presto! A script is magically produced, and your test is automated. What could be easier?
Manual testing was easier, as it turns out, because no one had factored in the extra coding to make the scripts actually work and the maintenance overhead. Customers quickly discovered that it took more effort to keep their scripts current than to revert to manual execution. Add in the expense of specialized scripting skills, and automation cost more, too.
What makes this an even bigger tragedy, aside from the wasted money and effort, is that the pace of application development has continued to accelerate and the associated risk of failure has grown exponentially. Test coverage has never been more important and yet has never been lower. Manual testing simply cannot keep up.
So, here we find ourselves in the new millennium, trying once again to convince management that automation is the answer, but with new technologies that address the shortcomings of the now—discredited capture/playback tools by reducing or removing the need for script code and related skills and minimizing the inevitable maintenance. The problem is that they have heard this song before.What to Do?
As is true with so many other recovery programs, the first step is to admit you have a problem. No more dancing around or outright denial: If most of your testing is still manual, confess it. That doesn't mean you have to fall on your sword; management must be made to understand that the promise of automation was built on a faulty premise about what it would take, but not what it would return. Believe it or not, there are still many who haven't heard the news—announced on this very site and elsewhere—that capture/playback doesn't work and never did. The key here is to convince management that you learned your lessons from the first attempt and won't repeat them.
Next, you have to propose a practical, provable, and attractive alternative—practical in that you should pick a project and make it successful before planning an enterprise rollout, and provable in the sense of being carefully measured to confirm that success. Attractive is all about the business case; done right, automation can and does yield amazing returns. But don't sugarcoat the costs. Be up front about what it will take in terms of time, people, and money, in addition to the licensing fees for new technologies. Automation is worth it, but you have to make the investment.
Finally, you have to execute. This means not only conducting a thorough evaluation or proof of concept but also maintaining a healthy skepticism throughout the process. Actively question promises that seem too good to be true. Don't settle for the one or two references the vendor gives you. Ask around, and insist on talking to those who have weathered changes in versions, personnel, and technology to uncover the hidden costs of maintenance and skills required to keep the automation running.
You may wonder if it is even worth it to try again, and the answer is that ultimately you don't have a choice. If you don't make automation work, someone else will, because all those billions are an indisputable testament to the fact that management does, in fact, value automation. They may have been fooled about how to go about it, but not about the fact that they need it—now more than ever.
