|
|
Achieving Meaningful Metrics from Your Test Automation Tools In addition to the efficiency improvements you expect from automated testing tools, you can-and should-expect them to provide valuable metrics to help manage your testing effort. By exploiting the programmability of automation tools, you can support the measurement and reporting aspects of your department. Learn how Jack Frank employs these tools with minimal effort to create test execution
status reports, coverage metrics, and other key management reports. Learn what measurement data your automation tool needs to log for later reporting. See examples of the operational reports his automation tools generate, including run/re-run/not run, pass/fail, percent complete, and percent of overall system tested. Take with you examples of senior management reports, including Jack's favorite, "My Bosses' Boss Test Status Report"-names will be changed to hide the guilty. Regardless of the
|
Jack Frank, Mosaic Inc
|
|
|
Structured Testing within the Rational Unified Process Many organizations have adopted, or are in the process of adopting, the Unified Process (UP) and, in particular, the Rational Unified Process (RUP). The test process defined within UP/RUP differs from more traditional, structured testing processes such as TMap (Test Management Approach) in Europe and STEP™ (Systematic Test and Evaluation Process) in the US. Tim Koomen, who has operated within these and other development lifecycle and test processes,
describes testing as defined in UP/RUP, maps the processes to those in TMap, and combines them into a "best of both worlds" approach. Learn about the UP/RUP defined practices such as the risk based test strategy, testability, test design, the role of the tester, independent testing,
|
Tim Koomen, Sogeti - Netherlands
|
|
|
Can You Find Bugs in Your Pajamas? Becoming an Effective Telecommuting Tester Distributed development teams, including test engineers, are becoming more the norm than the
exception. Many individual testers and test managers perform some of their job duties from
home. Test engineer Andy Roth is an extreme example of this situation-telecommuting from his
Maryland home 300 miles from his company’s office. As a “tele-tester” Andy has become a
manager in addition to his testing duties, managing his personal test lab, his time, his peer
relationships, and even managing his manager. If you are considering becoming a tele-tester,
already are one, or manage tele-testers, join Andy for a discussion of what it takes to survive and
flourish in this environment. Find out the necessary prerequisites and qualities of successful teletesters
and the tools of the trade that make life easier and most productive.
- The case for tele-testing and its limitations
|
Andy Roth, IBM Rational Software
|
|
|
The QA/Testing Perspective on Software Security Most everyone now realizes that we cannot solve security vulnerabilities with firewalls, virus scanners, and other tactics that build an electronic “moat” around systems. According to Julian Harty, security is not an operational issue, not a developer issue, and not a testing issue. It is a systems issue that you must focus on throughout the software’s life. From a QA/testing perspective, we need to look early in the development process for adequate security requirements. Then, we should assess the designs for vulnerabilities and participate in security code reviews. When specialized, security tests find bugs that get past our early prevention efforts, causal analysis helps prevent the recurring security defects. Dig into system security issues with Julian and learn about manual techniques, commercial software, and home-brew automation tools to help you find security vulnerabilities-before the bad guys do.
|
Julian Harty, Commercetest Limited
|
|
|
A New Paradigm for Collecting and Interpreting Bug Metrics Many software test organizations count bugs; however, most do not derive much value from the practice, and other metrics can actually harm the quality of their software or their organization. Although valuable insights can be gained from examining find and fix rates or by graphing open bugs over time, you can be more easily fooled than informed by such metrics. Metrics used for control instead of inquiry tend to promote dysfunctional behavior whenever people know they are being measured. In this session, James Bach examines the subtleties of bug metrics analysis and shows examples of both helpful and misleading metrics from actual projects. Instead of the well-known Goal/Question/Metric paradigm, James presents a less intrusive approach to measurement that he describes as the Observe/Inquire/Model. Learn about the dynamics and dangers of measurement and a new approach to improve your metrics and the software you produce.
|
James Bach, Satisfice Inc
|
|
|
Testing Software Builds Automatically Using Virtual PC Software When they receive a new software build, testers usually start their automated testing runs. Instead, what if they could execute tests automatically at the end of the build process? Using Microsoft Virtual PC 2004 as the example tool, Geoff Stewart shares his experiences using virtual PC software to configure, baseline, and isolate a test environment and automatically execute tests as part of the build process. By using virtual machine software via a command line interface, anyone can run tests without knowledge of the testing tool. Geoff demonstrates the technique and explains how he saves test results in XML for easier post-processing and historical reference. Get a head start on test execution, eliminate version conflicts, and save time installing test tools on host machines.
- An overview of virtual machine technology
- Integrating test execution with an automated build process
|
Geoff Stewart, Itron Inc
|
|
|
Free Test Tools are Like a Box of Chocolates You never know what you are going to get! Until you explore, it can be hard to tell whether a free, shareware, or open source tool is an abandoned and poorly documented research project or a robust powerhouse of a tool. In this information-filled presentation, Danny Faught shows you where open source and freeware tools fit within the overall test tool landscape. During this double session, Danny installs and tries out several tools right on the spot and shares tips on how to evaluate tools you find on the Web. Find out about licensing, maintenance, documentation, Web forums, bugs, and more. Discover the many different types of testing tools that are available for free and where to find them. Danny demonstrates examples of tools that you can put to use as soon as you get back to the office.
|
Danny Faught, Tejas Software Consulting
|
|
|
Using Personas to Improve Testing Too often testers are thrown into the testing process without direct knowledge of the customers' behaviors and business process. As a tester, you need to think and act like a customer to make sure the software does-in an easy-to-use way-what the customer expects. By defining personas and using them to model the way real customers will use the software, you can have the complete customer view in designing test cases. Get the basics of how to implement customer personas, their limitations, and ways to create tests using them. See examples of good bugs found using personas while learning to write bug reports based on them.
- What you need to know to develop customer personas
- Use customer personas for designing test cases
- The types of bugs found by using personas but missed by other techniques
|
Robyn Edgar, Microsoft
|
|
|
A "Follow the Sun" Test Automation Strategy In this case study of an award winning project, Andy Redwood describes how his team used "best shoring" of testing services to reduce costs, reuse assets, and get the best from their test automation tools. In an enterprise-wide transformation process at a large investment bank, his team used available infrastructure, technology, tools, and process to reduce business risk from software changes with a new automated regression test suite. With some facts and figures and a little hindsight, you will learn how to provide global, automated testing services on a twenty-four hours a day/seven days a week, on-demand basis. Find out what metrics you need to accurately measure the costs and benefits of a "follow the sun" test automation strategy.
- A successful outsource project that measurably improved business resilience
- The do's and don'ts of offshore testing
|
Andy Redwood, Buttonwood Tree Group
|
|
|
Conveying Performance Test Results with Data Visualization Performance and load testing generate barrels of data about transaction times, hardware resource utilization, and system behaviors. Your job as a performance engineer is not only to know how to use automated tools to create this data, but also to summarize, interpret, draw conclusions, and effectively communicate the results of tests. Do your job well, and management will be compelled to act on your recommendations. Dan Downing describes the tools and techniques he regularly employs for creating displays and graphs that visually correlate complex application performance and system resources metrics, turning a mass of data into concise and powerful information. Participants will walk away with specific techniques for using Excel and commercial or open source load testing tools to develop and clearly communicate performance test results.
- The results graphs that support performance/load testing objectives
|
Dan Downing, Mentora Inc
|
