Ten Frontiers for Software Testing

October marked the one-year anniversary of the call for participation for the first Writing About Testing (WAT) conference. Ultimately, fourteen of the best software testers in the world (along with a representative from StickyMinds.com) met in Durango, Colorado, for two days to discuss the state of public discourse in software testing.

I proposed the WAT conference because what we know as "software testing" risks not keeping up with the overall state of the art of software development. The fundamental positions on subjects like certification, heuristics, exploratory testing, open source, automation, etc. were all formed long ago, and the popular discussion on the subject of software testing risks becoming an echo chamber where the same concepts go around and around forever. But, I know that there is some wonderful and creative software testing going on in the world.

At the conference, we had presentations and discussions on REST architectures (where the concept of a "user interface" becomes very fluid), data visualization, system administration, programming, creating test frameworks as mashups of existing applications, abstractions based on agile methods, the role of aesthetics and artistic performance in software development, and a wealth of other topics. My goal for the WAT conference was simply to stretch out and make room to work with some of these new concepts. For the second WAT conference, I want to expand the software testing discourse into newer areas. To that end, I propose ten frontiers for software testing:

System Administration
There is a surge of interest in recent times in a concept called DevOps, which proposes an alliance among software developers and system administrators in order to create the best possible production environment experiences. Testers need to be a part of that conversation. Testing system upgrades, testing deployments, and using virtual machines and cloud resources in order to know in advance what will happen on production systems is interesting work that gets very little exposure.

Data Visualization
Not only our applications but also the whole world around us generates incomprehensible amounts of data, and the only way to make sense of it all is to render that data in a visual or tactile fashion. Making maps of territories, explicitly showing hidden connections among sets of otherwise obscure data, and tracking information about development projects in coherent ways are all relevant to testing software.

Frameworks/Mashups
Good test automation today happens at every level. A single framework may exercise the user interface, call REST or SOAP APIs, and reach into a database—all in the course of a single test suite. Myriad tools for such testing exist. Finding and choosing these tools—and knowing how to get such tools to talk to each other for a particular purpose—are becoming critical skills for testers.

User Experience
Great strides have happened in user experience work in the past few years, and there are exciting advances on the horizon. From touch screens to efficient navigation paths in applications to feature design, the community of software creators is rethinking approaches to engage users with software. Testers have largely ignored the conversation happening among user experience experts, but testing expertise can inform that conversation.

Web Design
New JavaScript libraries like jQuery are making things possible in browsers that were unimaginable just a few years ago. Adobe’s Flex, Flash, and AIR are making huge inroads into application UI design by bringing rich content to desktops and to web browsers without the need for heavyweight installations or coding. HTML5 is on the horizon and looming. Testers need to know what is happening in this arena and to understand the changes such technologies will bring to their tools and the way work is approached.

Web Services (REST/SOAP)
Twitter, Facebook, and the bleeding edge of web applications are no longer about the UI. Today, it is all about the APIs and the third-party applications that use those APIs to bring killer experiences to users. Testers need to know how web services and APIs work. Architectures based on exposing APIs are evolving quickly, and testing those architectures is changing the way we think about subjects like integration testing and the testing of business functions.

Environments, Virtual Machines, and Cloud Computing
Managing test environments has always been challenging. New cloud computing services, distributed systems, and virtual machines in some ways make this sort of testing work more challenging, but the reward is a vastly simplified process for provisioning test environments. Cloud computing services are also fueling innovative software startup companies, and cheap, virtual machines allow testers to work in systems that would have been radically expensive just a few years ago.

Agile Methods
Agile methods work, but even today no one knows exactly why. That such a simple set of rules could have such a profound effect on the nature of software development is rather strange. Testers could be the ones to provide the well-considered explanations for the effectiveness of agile methods.

Process Work/Quality Assurance
Quality assurance (QA) has a bad reputation in the testing community that it does not deserve. I have said before on StickyMinds.com and in my chapter in the book Beautiful Testing that QA is not evil, that it is work that still needs doing, and that testers often are in a good position to provide QA. QA is about process and new approaches like acceptance-test-driven development, behavior-driven development, DevOps, continuous deployment, etc. These approaches are changing how the world thinks about software development process. Bring back real discussion about QA.

Aesthetics/Artistic Performance
There is a wealth of knowledge available from disciplines within the liberal arts that applies directly to software development. Testers can help bring that knowledge over to the world of software development. For example, the Australia/New Zealand Weekend Testers recently had an exercise in “close reading” that was fascinating for someone like me with a background in critical theory. Also, I and number of other voices in the software community relate our work in academia and as artists to our work in software.

I suspect these ten frontiers are only a fraction of the sorts of new and interesting work being done by software testers out in the world. I offer them as the start of a public conversation, hoping to advance the state of the practice of software testing, and the state of the art of software development. Feel free to add more ideas for software testing frontiers in the comments below.