Conference Presentations

Performance Testing Throughout the Life Cycle

Even though it is easy to say that you should continuously test your application for performance during development, how do you really do it? What are the processes for testing performance early and often? What kinds of problems will you find at the different stages? Chris Patterson shares the tools and techniques he recently used during the development of a highly concurrent and highly scalable server that is shipping soon. Chris explores how developers and testers used common tools and frameworks to accelerate the start of performance testing during product development. Explore the challenges they faced while testing a version 1 product, including defining appropriate performance and scale goals, simulating concurrent user access patterns, and generating a real world data set. Learn from his team's mistakes and their successes as Chris shares both the good and the bad of the process and results.

Chris Patterson, Microsoft
I Wouldn't Have Seen It If I Hadn't Believed It: Confirmation Bias in Testing

"It ain't what we don't know that gives us trouble; it's what we know that ain't so." Will Rogers was talking about confirmation bias-the tendency to feel secure in our beliefs rather than to seek evidence that might challenge them. In testing, confirmation bias prompts us to stop a test too early, to choose tests that conform too closely to the happy path, or to ignore results that confound our expectations. As a result, defects have a chance to hide in our self-induced blind spots. We can't eliminate confirmation bias, but we can manage and control it by diversifying our models, our techniques, and our test teams. In this hands-on and eyes-on session, Michael Bolton presents a set of exercises, videos, and conversations that show testing biases in action. Discover some new tricks that can help you defend yourself and your testing clients from being too sure, too soon, and later ... sorry.

Michael Bolton, DevelopSense
A Test Odyssey: Building a High Performance, Distributed Team

It seemed simple enough-hire the best available technical staff that would work from home to build some great software. Along the way, the team encountered the usual problems: time zone differences, communication headaches, and a surprising regression test monster. Matt Heusser describes how Socialtext built their high-performance development and test team, got the right people on the bus, built a culture of "assume good intent and then just do it," created the infrastructure to enable remote work, and employed a lightweight yet accountable process. Of course, the story has the impossible deadlines, conflicting expectations, unclear roles, and everything you'd get in many development projects. Matt shares how the team cut through the noise, including building a test framework integrated into the product, to achieve their product and quality aims.

Matthew Heusser, Socialtext
The Elusive Tester-Developer Ratio

Perhaps the most sought after and least understood metric in software testing is the ratio of testers to developers. Many people are interested in learning the standard industry ratio so that they can determine the proper size of their test organization. Randy Rice presents the results of his recent research on this metric and explores the wide range of tester-developer ratios in organizations worldwide. Learn why this metric is almost always not the best way to determine your test organization’s staffing levels and how to understand and apply this metric in more helpful ways. Find out how different tester-developer ratios relate to test effectiveness. Take away a new appreciation of your own tester-developer ratio and ways to meaningfully convey this metric to management to help rightsize your test team and improve the ROI of testing. Determine the "right ratio" of testers to developers in your team and company.

Randy Rice, Rice Consulting Services
Virtualizing Overutilized Systems to Eliminate Testing Constraints

Organizations currently are using virtualization in the test lab to eliminate underutilized systems such as physical computers and software. So why not virtualize the costly, overutilized, or completely unavailable elements of the software architecture that have serious access and data issues for testing? These elements required for realistic end-to-end testing-mainframe computers, production systems of record, and computing services hosted by other companies-are often difficult or expensive to access for testing. Rajeev Gupta explains how virtualizing these overutilized systems can make the constraints of capacity, test data, and availability for testing a distant memory. Discover how service virtualization, employed as an adjunct to hardware lab virtualization, eliminates the bottlenecks and data management efforts that stymie many test and development teams.

Ken Ahrens, iTKO
Executable Specs w/ FitNesse Selenium

"Executable Specifications with FitNesse and Selenium."

Dawn Cannan, DocSite LLC
Meet "Ellen": Improving Software Quality through Personas

Users are the ultimate judge of the software we deliver because it is critical to their success and the success of their business. However, as a tester, do you really understand their tasks, skills, motivation, and work style? Are you delivering software that matches their needs and capabilities-or yours? Personas are a way to define user roles-imaginary characters-that represent common sets of characteristics of different users. David shares how his team at Microsoft defined and used one persona named “Ellen” to help them design, develop, and test the first version of a new product. David shares before Ellen and after Ellen examples of the product, showing how the product changed when Ellen joined the team. See examples of the robust test cases and acceptance scenarios they defined from unique insights that Ellen provided.

David Elizondo, Microsoft Corporation
STAREAST 2010: Service-driven Test Management

Over the years, the test manager's role has evolved from "struggling to get involved early" to today's more common "indispensable partner in project success." In the past, when "us vs. them" thinking was common, it was easy to complain that the testing effort could not be carried out as planned due to insufficient specs, not enough people, late and incomplete delivery, no appropriate environments, no tools, tremendous time pressure, etc. Martin Pol explains how today's test managers must focus on providing a high level of performance. By using a service-driven test management approach, test managers support and enhance product development, enabling the project team to improve overall quality and find solutions for any testing problem that could negatively impact the project's success.

Martin Pol, POLTEQ IT Services BV
Chartering the Course: Guiding Exploratory Testing

Charters help you guide and focus exploratory testing. Well-formed charters help testers find defects that matter and provide vital information to stakeholders about the quality and state of the software under test. Rob Sabourin shares his experiences defining different exploratory testing charters for a diverse group of test projects. For example, reconnaissance charters focus on discovering application features, functions, and capabilities; failure mode charters explore what happens to applications when something goes wrong. In addition, you can base charters on what systems do for users, what users do with systems, or simply the requirements, design, or code. Rob reviews key elements of a well-formed testing charter-its mission, purpose, focus, understanding, and scope. Learn how to evolve a test idea into an exploratory charter using examples from systems testing, Scrum story testing, and developer unit testing.

Robert Sabourin, AmiBug.com
Test Automation Success: Choosing the Right People and Process

Many testing organizations mistakenly declare success when they first introduce test automation into an application or system. However, the true measure of success is sustaining and growing the automation suite over time. You need to develop and implement a flexible process, and engage knowledgeable testers and automation engineers. Kiran Pyneni describes Aetna’s two-team automation structure, the functions that each group performs, and how their collaborative efforts provide for the most efficient test automation. Kiran explains how to seamlessly integrate your test automation lifecycle with your software development lifecycle. He shares specific details on how Aetna’s automation lifecycle benefits their entire IT department and organization, and the measurements they use to track and report progress.

Kiran Pyneni, Aetna, Inc.

Pages

CMCrossroads is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.