|
|
The Four T's of Test Automation Historically, organizations have measured the level of software quality after a system goes into production. What about measuring the level of software quality coming into testing? By measuring the number of test cases that pass or fail, and calculating the failure rate, you can forecast future failure rates and the number of test cases that will have to be executed each day during test. Then, you will have the information needed to adjust resources, timelines, and level of effort. Join Jan Fish to learn how to manage and control your testing effort rather than having it control you. By tracking the number and severity of bugs by build, you can open a whole new world of information. Combining this metric with knowledge of the content of the build lets you see immediately if newly introduced functionality is stable, if it disrupts surrounding functionality, or if your test effort should be modified.
|
David Dang, Questcon Technologies, A Division of Howard Systems Intl.
|
|
|
Test Design for Complex System Testing While functional testing generally focuses on specific areas of a single component, system testing focuses on multiple components and interactions within their environment. As software becomes more distributed and their topologies become more complex, these interactions require testers to apply additional skills in test design and execution. Mark Duquette offers an overview of system testing for those who are experienced in functional testing and are looking for more information about complex system testing. Mark introduces several types of system tests and explains how they differ. He explains the roles of workloads, topologies, data, and metrics as they relate to designing test cases for complex situations. Learn how to create reusable test cases for comparing quality over multiple releases. Explore using transaction profiles to ensure realistic workloads for performance tests.
|
Mark Duquette, IBM
|
|
|
How to Build and Motivate the Ideal Test Team Testing projects have a habit of getting into trouble. After years of waterfall development in which testing is the last stage in the development pipeline, there are almost always deadline and budget squeezes that require test managers to attempt to do the impossible. Most of us have had to manage risks as part of the test management process. However, the most plausible mitigation strategy is not always the best one. Geoff Horne presents a method he uses for identifying and assessing risks and then developing mitigation strategies for testing projects. Geoff has successfully used the approach on different projects across different types of businesses and testing projects. His approach is based on evaluation of risks and assessing the impacts across the key criteria of resources, productivity, cost, quality, and confidence.
|
Lloyd Roden, Grove Consultants
|
|
|
Today's Testing Innovations As a consultant, Lee Copeland has spoken with thousands of software testers in hundreds of different organizations. Generally, he comes away from these discussions depressed with the state of testing. Many organizations neither know about nor have adopted recent important innovations in our field. Lee will discuss nine of the important innovations in testing-the context-driven school, testing specialties, test-first development, really good books, open source tools, session-based test management, testing workshops, certification, and freedom of the press. Join Lee for his list, and propose others if you'd like. Discuss the keys to innovation and take a test evaluating your organization's innovation quota.
- The top nine innovations in testing today, plus the ones you add
- How to be more innovative in your testing
- Evaluate your organization's openness to innovation
|
Lee Copeland, Software Quality Engineering
|
|
|
Practical Pairwise Testing with PICT Fault analysis reveals that interaction between the variables of dependent parameters is a common source of failure in complex systems. Imagine you are assigned to test a feature with twenty independent parameters and five possible states for each parameter. The total number of possible combinations is greater than five-hundred billion. At one test executed per millisecond, it would take more than 3,000 years to test all possible combinations. So, which combinations do we test? Pairwise testing is a systematic procedure to reduce the total number of tests by selecting a set of tests that evaluates every pair, rather than every combination. BJ Rollison compares the orthogonal arrays method to pairwise analysis and provides a detailed example of how to use PICT, a powerful, highly configurable combinatorial analysis tool that is freely available.
|
Bj Rollison, Microsoft Corporation
|
|
|
The Promise of Model-Based Testing Good test design is the cornerstone of every test effort. Although approximately 50% of testing time is spent on test design, very little has been done to bring structure, automation, and a scientific approach to the test design process itself. The quality of designs is highly dependent on the individual tester's expertise. Mahesh Velliyur N has found that model-based testing is extremely useful in testing end-to-end business scenarios. Model-based testing brings a rigor to test design by focusing on what and how much to test. Models are useful in testing only when they are able to generate a useful set of test cases providing measurable coverage and traceability back to the system under test. Often, model-based testing meets those requirements. In addition, it avoids duplication of tests at different test levels, ensures more complete data coverage, and helps in prioritizing test execution.
|
Mahesh Velliyur, Maveric Testing Solutions
|
|
|
Test Estimation: Simple Models and Practical Lessons As software testers, we are regularly asked how long it will take to test a system. Unfortunately, we rarely have the tools to produce an accurate estimate. Tonnvane Wiswell introduces methods for producing better estimates-best guess, experienced person's best guess, and ways to use past data as a baseline--and the advantages and disadvantages of each. She discusses adaptable formulas that incorporate "buffer time" and risk factors. Finally, Tonnvane presents a real life example of a testing project with solid time estimates, including an explanation of how team size was determined, how the work flow was designed, what the "actual hours" of testing were, what unexpected items affected the testing time, and how the project permanently changed the company's "test estimation formula."
- Two simple estimations methods when you don't have historical data
- How to improve test estimation in your organization
|
TONNVANE WISWELL, Total Jobs Group
|
|
|
Five Steps to Becoming a Top Tester Have you ever wondered what top testers do that enables them to accomplish so much more than an average tester? As a tester, the key to maximizing your potential for success is taking responsibility for developing your own testing skills. Bernie Berger shares five testing tips that can help you strengthen your testing ability and set you apart from the crowd. Learn what it means to know your stakeholders; why it is important to think concurrently; where to look for boundaries in testing your application; how to test beyond the specifications; and when to be skeptical. The common theme of these tips is: If you want to improve the way you test, you first have to improve the way you think. Bernie believes that testing is more than just a job--it is a cool way to earn a living, and we can all do it better.
- The hidden stakeholders in your system under test
|
Bernie Berger, Liquidnet Holdings, Inc.
|
|
|
Pairwise Testing Comes of Age You've heard of orthogonal arrays and pairwise testing. Perhaps you've used a pairwise test case generator tool. Have you ever wondered where these popular and powerful techniques originated? Actually they have been around for almost twenty years. During this time, important test design principles have emerged and choices for test generation tools have improved. George Sherwood, inventor of CATS, one of the first pairwise test tools, reviews what we have learned and how it applies to testing today. He shows the benefits of using pairwise test techniques for selecting configurations and for generating test data. George also outlines important considerations for successful pairwise test designs, including the problems to anticipate and avoid. George dispels the mystery of pairwise test generators with a simplified view of how they work.
|
George Sherwood, Testcover.com
|
|
|
STAREAST 2008: Telling Your Exploratory Story What do you say when your manager asks you, "How did it go today?" If you have a pile of test cases on your desk, it may be acceptable for you to say, "I ran x% of these tests today," or "I'll be finished with this stack in y days at the rate I'm going." However, if you're using exploratory testing as your approach, it may be downright terrifying to try to give a status report, especially if project stakeholders think exploratory testing is irresponsible and downright reckless compared to pre-scripted test cases. So how, then, can you retain the freedom and power of exploration, and yet give a report that earns you credibility, respect, and perhaps more autonomy? Jon Bach suggests ways to describe and explain the critical and creative thinking that drives your exploratory testing so that others have a better chance of understanding and appreciating your value as an exploratory tester.
|
Jon Bach, Quardev, Inc.
|
