Is your testing tool behaving like a problem child? Injecting your application with some discipline may be what's needed. Sometimes, spending a few weeks of development time on creating a solution can save you months of testing time. In this article, Linda Hayes explains how one company did just that!
One of the thorniest problems in test automation is dealing with objects that the test tool can't see or interact with. Often these are third-party controls, but not always. Sometimes they are just complex controls within containers or other layers that obscure access to the methods and properties. But whatever the reason, in my experience they make up 20 percent of the application but take 80 percent of the effort, and in some cases, they have stymied automation altogether.
One of the most effective ways of dealing with this issue is to inject the application with some code or hook that gives the test tool access to the application internals. This might take the form of a DLL that is compiled in, an API that is exposed, or special methods that are added. But while this approach is usually the most powerful and efficient solution, it is usually the least likely to succeed for the simple reason that developers don't want to modify the application for test purposes.
The reasons run the gamut from an inchoate fear that the extra code might cause unpredictable problems, to the belief that if the test hook is removed when the application is delivered that the "real" code wasn't really tested. If it stays in, there is a major security risk that others might use it to interfere with the application. Or, in some cases, the developers simply don't see it as their problem, or they aren't motivated to invest the time and effort.
It seems there are no easy answers—or at least it did seem that way, until an enterprising software development company called CSG Systems came up with an ingenious solution. CSG Systems has been providing customer care and billing applications for twenty years to 265 companies in forty countries. Their software portfolio is massive, complex and critical—a perfect candidate for test automation.
The Problem
Like most companies dealing with huge applications and widely varied customer needs, CSG has adopted an architecture that permits flexibility through configurability. This design relies on multiple components that are shared, resulting in objects that are contained with other objects—a classic automation challenge.
"We had objects that the test tool could not see," said Senior Manager Shane Perrien. "This prevented us from accessing the object methods and properties we needed." A related issue was the dynamic nature of the application: different conditions created different window object contents, and since the window handle was created new at runtime, it was difficult to set context. This caused excess maintenance overhead for test cases.
After exhausting all other options, Shane presented the problem to the development organization. Software Architect Miao Chen developed an application DLL that exposed the methods and properties of each window and object, and a second DLL stored in the system directory that accesses the first for the automated engine. Miao then created a tool for QA to identify a unique path for each object. Once these were in place, Edmond Sierens, the Software Engineer responsible for the test automation function library, retrofit his functions to call the system DLL and expose the window handle, which is then sent back with the unique object ID.The Solution
CSG could now effectively and thoroughly automate their test execution as well as efficiently manage and maintain dynamic content. The test tool could call the system DLL for a particular object, and the system DLL could call the application test DLL to retrieve or set the methods and properties.
While this sounds complicated, it's actually very elegant, because it permits the application to be delivered exactly as it was tested, with one small difference: the test DLLs are not part of the package. In other words, the application software does not have to be recompiled because the test hook is not embedded within it. And, since the test DLLs are not delivered, the special functions they provide are not available, thus sealing any potential security hole. "We have delivered two releases with the test DLLs removed and have not experienced any issues," said Shane.
The test automation team now has full, reliable access to all application objects and can shift their time and attention to the real task: developing an extensive automated test library that can deliver comprehensive coverage of their applications in a timely and effective manner. By improving their efficiency, the test team has more time to invest in analysis and design to continually improve quality. Customers receive better software faster, developers have fewer fixes in the field—everyone wins.
The Effort
So what was the price of all this? Six weeks. One week for the application DLL, two weeks for the mapper, and three weeks to retrofit the automation function library. And how did the QA organization get the cooperation from development?
"We know that it takes teamwork to deliver a quality product," said John Klimek, Executive Director of QA. "If spending six weeks of development can save months of testing time, then it's an easy decision to make."
What a breath of fresh air! Too many companies view development and QA as different—if not adversarial—functions. You and I know that the opposite is true. They are integral to each other, each playing a vital role in delivering good software. When I hear that developers can't be bothered to invest time in improving testability, I am reminded of a cartoon where one person in a boat gloated to the other, "Your end of the boat is sinking."
So is CSG an anomaly, or are there other examples of encouragement you can tell me about out there?
