Load testing is a necessary but daunting task on many testers' to-do lists. As difficult as the task may be, it helps testers find problems when a software product is running at full capacity. Danny R. Faught explains how he takes an exploratory approach to load testing, which gives quick results without making a big tool investment.
Load testing often involves a complex tool with a steep learning curve, possibly costing many thousands of dollars and a big investment in hardware to drive the load. I have learned how to sidestep these complications--at least at the beginning of a load testing effort.
I like to call my approach "exploratory load testing." Most of what is written about exploratory testing is in the realm of simple functional testing, but the same concepts can apply to load testing. To round it out, I throw in agile automation practices. You can do exploratory load testing using one of the full-featured commercial or open source load test tools, but I find that I'm more effective when crafting tests in a general-purpose scripting language using the available open source programming libraries. And that's the approach that I'll discuss here.
Exploratory load testing is most useful when the organization has not put much thought into the performance requirements of the system and hasn't developed a statistical usage profile. I'm usually asked to find the limits of the software, and then management decides which limits need to be raised. Often, in my experience, the exploratory load tests are all the load testing the project needs, but you may find that you need to employ a more sophisticated performance and load test after the product matures.
Dipping Your Toe in the Water
You'll need to do some programming in order to do load testing well. If you're not yet a programmer, take heart, because you can build your skills incrementally. On one project, I needed to load test an application that does peer-to-peer networking. The simplest thing I could start with was to get as many of my old junker computers as possible up and running, and manually run one instance of the application on each of them. Four or five computers running the application were enough to shake out some basic functional problems that would be showstoppers for a more elaborate load test.
As the application became more reliable, I needed to run more instances of it. I asked the development team to add features that would allow me to run more than one instance of the application on one computer. Management supported this effort, because the alternative would be to buy scores of additional computers. I was able to run between five and thirty copies of the application on each of my test machines, depending on the hardware resources on the machine, with a total of up to a hundred copies running in my test lab. Simply starting the applications and checking to see if they all saw each other on the network was a sufficient test for finding several load-related bugs.
It didn't take long before the tedium of manually starting the applications prompted me to start writing a tool to take over the task. I wrote a short bash shell script that launched a specified number of instances of the application on a machine, and another one that shut down all of them. I could then issue one command on each of the test machines to kick off the load test.
Building a Web Robot
On another project, I needed to conduct a load test for a Web application within a short time frame. After learning the basics of how the Web site worked, I had about half a day to build and run a load test to give a preliminary assessment of how reliable the application was. This time I used Perl and the WWW::Mechanize library to simulate a Web browser. I paired with another tester who had experience using commercial load test tools but who wasn't familiar with Perl. It took forty-two lines of code to log in a user and click a link and iterate as many times as we desired. We only had three test accounts available, but once we proved that a user could log in from more than one browser at a time, we programmed the script to repeatedly log in the same few users.
At first I "drove," writing the preliminary version of the script and showing my partner how I used the automation library. When we found that we could log in thousands of sessions at a time without error, my partner took over the keyboard. We added code to randomly choose sessions that had already logged in and had them click another link to keep the session from timing out.
This was probably my first project where the first simplistic approach to load testing did not find any serious limitations in the product. With further exploratory functional testing, we did find numerous robustness problems, such as unexpected form input causing server exceptions. So after a few days of testing, we were able to report that robustness problems were more of a concern than the reliability of the system under load. We still needed to follow up with a more elaborate load testing approach before we could confidently say we had found most of the major reliability issues. Continuing the incremental approach to load testing would ensure that the project gets timely feedback on reliability, so that management can easily decide whether to invest in additional load testing.
Organic Script Gardening
In the Web testing example above, the script was contained in a single file. After a few hours of development, we started to abstract some common actions into subroutines. This was sufficient for what we needed for the short project.
For the peer-to-peer application discussed above, I was involved in the testing for several months and developed a whole suite of test tools for it. The application uses several different network protocols. When I started experimenting with testing a new protocol, my test code would be contained within a single script. As the script became more sophisticated, I would extract code into subroutines and eventually move the subroutines in an object-oriented test library. The code grew organically, and I carefully tended the developing garden of code so it didn't grow wild and unmanageable. I now have a powerful and continually growing arsenal of application-specific libraries that help me to quickly craft variations on the load tests I've done before.
The good news for you is that you can start to build load tests using an incremental approach. Adjust your approaches as you learn more about how the system reacts. Once you find one bug, you can often tune your test to avoid that bug. This way, you can find additional problems without waiting for a bug fix. Choose whichever tool will get you started fastest, whether it's a full-featured load test tool or a scripting language with a collection of existing libraries.
References
