Conference Presentations

Performance Testing Throughout the Life Cycle

Even though it is easy to say that you should continuously test your application for performance during development, how do you really do it? What are the processes for testing performance early and often? What kinds of problems will you find at the different stages? Chris Patterson shares the tools and techniques he recently used during the development of a highly concurrent and highly scalable server that is shipping soon. Chris explores how developers and testers used common tools and frameworks to accelerate the start of performance testing during product development. Explore the challenges they faced while testing a version 1 product, including defining appropriate performance and scale goals, simulating concurrent user access patterns, and generating a real world data set. Learn from his team's mistakes and their successes as Chris shares both the good and the bad of the process and results.

Chris Patterson, Microsoft
Life as a Performance Tester

At the core of most performance testing challenges and failed performance testing projects are serious misunderstandings and miscommunications within the project team. Scott Barber and Dawn Haynes share approaches to overcoming some of the most common frustrations facing performance testers today. Rather than simply telling you how to improve understanding and communicate performance testing concepts, Scott and Dawn demonstrate their approaches through an amusing role play of interactions between a lead performance tester and a non-technical executive.

Scott Barber, PerfTestPlus, Inc.
STARWEST 2008: Performance Engineering: More Than Just Load Testing

Performance testing that is done once or a few times as part of the system test is not the right approach for many systems that must change and grow for years. Rex Black discusses a different approach--performance engineering--that is far more than performing load testing during the system test. Performance engineering takes a broad look at the environment, platforms, and development processes and how they affect a system's ability to perform at different load levels on different hardware and networks. While load testers run a test before product launch to alleviate performance concerns, performance engineers have a plan for conducting a series of performance tests throughout the development lifecycle and after deployment. A comprehensive performance methodology includes performance modeling, unit performance tests, infrastructure tuning, benchmark testing, code profiling, system validation testing, and production support.

Rex Black, QA Software Consultant/Trainer
Avoid Preformance Testing Data Deception

Don't be fooled by your performance test results. Performance testing can easily generate an unwieldy amount of data-some relevant and some not. Testers and their tools often use statistical methods to make sense of the data, but using statistics requires sacrificing accuracy and thoroughness. The good news is that we do not need to understand all the details to make good use of test results. The challenge is to determine what information really matters and how to present it in a useful manner. Join Ben Simo as he addresses common performance test statistical problems including built-in bias, agreeable averages, invisible inadequacies, gargantuan groupings, stingy sets, mountainous molehills, creative charting, alien alliances, and more. Find out how statistical reporting can deceive rather than inform-often unintentionally-and recognize what the numbers do not say.

Ben Simo, Standard & Poor's
Preformance Testing in Enterprise Application Environments

As systems become more complex--serving the enterprise and implemented on the Web and across the Internet-performance testing is becoming more important and more difficult. David Chadwick suggests that the starting point is to design tests that reflect real user activity, including independent arrivals of transactions and varying input data to prevent "cache only" results. David explains how to break down the end-to-end system response time into the distributed components involved in processing the transactions. Learn to use resource-monitoring data to discover bottlenecks on individual systems. By examining the frequency and time spent in various processes, performance testers can determine where resources are being consumed and how to tune a system for better performance.

David Chadwick, IBM
Load Generation Capabilities for Effective Performance Testing

To carry out performance testing of Web applications, you must ensure that sufficiently powerful hardware is available to generate load levels. At the same time, you need to avoid investing in unnecessarily expensive hardware "just to be sure." A valid model for estimating the load generation capabilities of performance testing tools on different hardware configurations will help you generate the load you need with the minimum hardware. Rajeev Joshi believes the models provided by most tool vendors are too simplistic for practical use. In fact, in addition to the hardware configuration, the load generation capabilities of any tool are a function of many factors: the number of users, frequency and time distribution of requests, data volume, and think time. Rajeev presents a model for the open source load generator tool, Jmeter, which you can adapt for any performance testing tool.

John Scarborough, Aztecsoft
Ten Indispensable Tips for Performance Testing

Whether you are inexperienced with performance testing or an experienced performance tester who is continuously researching ways to optimize your process and deliverables, this session is for you. Based on his experience with dozens of performance testing projects, Gary Coil discusses the ten indispensable tips that he believes will help ensure the success of any performance test. Find out ways to elicit and uncover the underlying performance requirements for the software-under-test. Learn the importance of a production-like test environment, and methods to create suitable environments without spending a fortune. Take back valuable tips on how to create representative workload--mix profiles that accurately simulate the expected production load. And more! Gary has developed and honed these practical and indispensable tips through many years of leading performance testing engagements.

Gary Coil, IBM Global Services
Preparing for the Madness: Load Testing the College Bracket Challenge

For the past two seasons, the Windows Live development team has run the Live.com College Bracket Challenge, which hosts brackets for scores of customers during the "March Madness" NCAA basketball tournament. March Madness is the busiest time of the year for most sports Web sites. So, how do you build your Web application and test it for scalability to potentially millions of customers? Ed Glas guides you through the process their team uses to model users, establish performance goals for their application, define test data, and construct realistic operational scenarios. Learn how the tests were conducted, the specific database performance and locking problems encountered, and how these problems were isolated and fixed. Finally, Ed demonstrates the custom reporting solution the team developed to report results to stakeholders.

  • How to establish performance goals and requirements
Eric Morris, Microsoft
Challenges in Performance Testing of AJAX Applications

The AJAX model for Web applications has been rapidly gaining in popularity because of its ability to bring the richness and responsiveness of desktop applications to the Web. Because one of the key drivers for the rapid adoption of AJAX is its promise of superior performance, it is surprising that there has been very little discussion of AJAX-specific performance testing. In fact, AJAX has a significant impact on aspects of the performance testing lifecycle including definition of goals, user modeling, and test scripting. Rajendra Gokhale discusses issues to consider: AJAX engine simulation and optimization, cross-client performance of AJAX applications, and design choices related to test scripting. Using Google's "Google Suggest" service as a case study, Rajendra examines the unique challenges of carrying out performance testing of AJAX-based applications and offers suggestions for overcoming them.

Rajendra Gokhale, Aztecsoft
Performance Testing Web Applications with OpenSTA

OpenSTA is a solid open-source testing tool that, when used effectively, fulfills the basic needs of performance testing of Web applications. Dan Downing introduces you to the basics of OpenSTA including downloading and installing the tool, using the Script Modeler to record and customize performance test scripts, defining load scenarios, running tests using Commander, capturing the results using Collector, interpreting the results, and exporting captured performance data into Excel for analysis and reporting. As with many open source tools, self-training is the rule. Support is provided not by a big vendor staff but by fellow practitioners via email. Learn how to find critical documentation that is often hidden in FAQs and discussion forum threads. If you are up to the support challenge, OpenSTA is an excellent alternative to other tools.

  • The capabilities and limitations of OpenSTA
Dan Downing, Mentora Inc

Pages

CMCrossroads is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.