How to Become a Better Performance Tester 101

People get into performance testing for various reasons, and some become very good at it over time, but some remain mediocre. Despite their usually disparate backgrounds, common merits among the best testers include broad knowledge of technologies in computer engineering, inquisitiveness and curiosity, and an avid pursuit of learning. If you are new to this field and want to get up to speed quickly, here are some tips.

Avoid Starting from Square Zero

For any release, we want to know whether the new code has performance impact. Before we set out to work on the actual tests, we generally need to collect information to determine areas that should be covered, scenarios that should be executed, load and performance acceptance criteria, etc.

But a common problem with inexperienced testers is that they do not know what questions to ask in order to get the most useful information. On top of that, if the developer has never been involved in performance testing before—which is not uncommon—he might not know what information to provide for your tests to effectively flush out performance issues. The combination of these two factors often leads to missing areas that should be tested or spending too much effort in areas that are less critical.

This is why, when possible, it’s critical for a junior performance tester to shadow someone who is more experienced for an extended period of time. With the right advice, he will pick things up more quickly and avoid many common pitfalls. Most importantly, he will be able to build the right mindset toward the general process of approaching a performance testing task from the very beginning. This is more important than learning to use a tool.

Other benefits include learning the tricks of using a tool, figuring out how to interpret test results (for example, if the response time of a transaction is reduced, does it really mean performance improved?), helping pinpoint performance bottlenecks, and becoming more skilled in test design.

Here, I want to stress that when I say a junior performance tester should shadow someone who is more experienced, I consider someone experienced if he or she has done requirement analysis, test planning, test design, scripting, test execution, problem solving, etc., for an extended period of time. This person must have a firm grip on the overall process—merely recording or running scripts for many years does not make one experienced.

How Can I Possibly Do All the Work?

Agile software development has been widely accepted these days, but not all companies can afford a dedicated performance testing personnel for every Scrum team; at least, that is the case for companies I have worked for—I always have to cover more than one team. If I attend all their meetings, there won’t be much time left for me to test. The strategies we typically adopt are:

• Get developers’ buy-in to do whatever they can to capture performance issues before delivering code to QA, such as unit testing and code reviews
• All teams agree to proactively reach out to us when they need to get a performance tester involved
• For teams that work on critical areas of the system or know beforehand performance testing will be needed, we go to their meetings more frequently
• Be as efficient and productive as possible
• Know the system as well as possible so we know where to focus our attention

These strategies worked most of the time, but there were times we had to scramble to meet the deadline when a performance issue popped out up and demanded a lot of our time. Also, it is worth noting that resource sharing is by no means a prescription for success in an agile environment. It should be minimized if possible.

In addition to the above strategies, it’s best to manage your time wisely. Try to plan and execute as early as possible, take the unexpected into account during planning, prioritize your work, and test business-critical areas and areas that often get hit the hardest first. 

Try to Learn at Least a Bit of Everything

The main difference between functional and performance testing is that functional testing concerns whether an application can successfully do something, while performance testing concerns how well the application can do it under load.

Modern systems typically consist of many components, each of which could have performance implications. Taking an extremely simplified system as an example, it probably has a database, an application, and a proxy, along with the networking, operating system, and firewall. Things could go wrong in any place in the system—inefficient code, hardware, network bandwidth, memory management, database table indexing, or competing threads, just to name a few. Therefore, the more you know about each component and how they interact with each other, the better you are equipped to design effective tests and help with diagnoses.

I once worked with a functional tester whose knowledge about the system under test exceeded that of even most of the developers. When she later started working on performance testing, that knowledge greatly compensated her lack of performance testing experience.

Often we have to stub part of a system in order to test another part because the stubbed piece is still under development or unavailable for testing. If you have a strong programming background and can write the stub yourself instead of waiting for development, you probably can get the testing going quicker.

How about some SQL? Performance testing normally requires a reasonable amount of diverse data for the results to be realistic. The quickest and most efficient way to prepare data is by inserting information directly into the database. If you can write SQL scripts instead of waiting for someone else to do it for you, wouldn’t that be fantastic?

What I am trying to say here is that broad knowledge about the technologies used in computer engineering can make things easier in every phase of performance testing: A Perl or Shell script can facilitate data analysis, networking knowledge can help diagnose performance issues, database knowledge can help analyze test results, and so forth. You don’t have to be an expert on all technologies involved because you can always learn when the need arises.

If I had to choose between someone with ten years of performance testing experience but no knowledge except the testing tool and someone else who is less experienced but knows a little bit of everything else, I probably will choose the latter. As a matter of fact, most of the good performance testers I know have backgrounds in fields other than testing.

Summary

Modern systems are complex, and performance requirements for each are unique. As a result, the approach for testing might be vastly different from one system to another. There is no magic bullet to quickly boost the efficiency and quality of your performance testing.

However, we implemented some of these suggestions in companies I have worked for and seen positive impacts. I hope you can do the same.