Manage the Business Risks of Application Development with Continuous Testing

Across virtually every industry today, software is increasingly becoming the interface to your business. Those organizations that are able to increase the speed and quality of innovative software releases will capitalize on differentiable competitive advantages; those that cannot will languish behind competitors.

Responding to the demand for more (and better) software faster, many enterprises have begun flirting with the idea of accelerating the software development lifecycle to drive innovation through software. In doing so, it’s critical to realize that there's an optimal balance between speed and quality with software delivery—as with all engineered products. We're in an era when leading organizations must reassess the true "cost of quality" for software. Remember that the cost of quality isn't the price of creating quality software—it’s the penalty or risk incurred by failing to deliver quality software.

Testing is the Elephant in the Room

As organizations start to accelerate the SDLC, process bottlenecks will become evident. One bottleneck that continues to plague SDLC acceleration is testing. At best, testing is considered inevitable overhead—a timeboxed event that occurs sometime between code complete and the target release date. At worst, organizations push quality processes to postrelease, forcing a “real-time customer acceptance test.”

Testing has long been the elephant in the room. The malleable nature of software has given organizations an excuse to defer investment in testing. However, it's important to recognize that this deferral results in technical debt. Over time, the technical debt compounds, escalating the risk and complexity associated with each release.

Another obstacle to SDLC acceleration is the lack of a coordinated, end-to-end quality process. If trustworthy and holistic quality data were collected throughout the SDLC, then more automated decision points could optimize downstream processes. Unfortunately, at the time of the critical “go/no-go” decision for a software release candidate, few organizations can confidently answer the questions:

• Are we done testing?
• Does the release candidate achieve expected quality standards?
• What are the quantifiable risks associated with the release candidate?
• How confident are we that we won't end up in the news?

This is why the concept of continuous testing is so critical.

How Continuous Testing Helps

Continuous testing provides a real-time, objective assessment of the business risks associated with an application under development. Applied uniformly, it allows both business and technical managers to make better trade-off decisions between release scope, time, and quality.

Continuous testing is not simply more automation. Rather, it is the reassessment of software quality practices, driven by an organization's cost of quality and balanced for speed and agility. Ultimately, continuous testing can provide a quantitative assessment of risk and produce actionable tasks that will help mitigate these risks before progressing to the next stage of the SDLC.

When it comes to software quality, we are confronting the need for true process reengineering. Continuous testing is not a "plug and play" solution. As with all process-driven initiatives, it requires the evolution of people, process, and technology. We must accommodate the creative nature of software development as a discipline, yet we must face the overwhelming fact that software permeates every aspect of the business—and software failure presents the single greatest risk to the organization.

The Business Value of Continuous Testing

Given the business expectations at each stage of the SDLC, continuous testing delivers a quantitative assessment of risk as well as actionable tasks that help mitigate these risks before they progress to the next stage of the SDLC. The goal is to eliminate meaningless tests and produce value-added tasks that truly drive the development organization toward a successful release.

Continuous testing—when executed correctly—delivers three major business benefits.

First, continuous tests are the artifacts that drive a central system of decision for the SDLC. Continuous tests place a bracer around core business functionality as expressed and implemented in software. The assessment of the validity or stability of these core business artifacts provides a real-time, quantifiable assessment of the application's health.

Second, continuous testing establishes a safety net that allows software developers to bring new features to market faster. With a trusted test suite ensuring the integrity of dependent application components and related functionality, developers can immediately assess the impact of code changes. This not only accelerates the rate of change, but also mitigates the risk of software defects reaching your customers.

Third, continuous testing allows managers to make better trade-off decisions. From the business perspective, achieving a differentiable competitive advantage by being first to market with innovative software drives shareholder value. Yet, software development is a complex endeavor. As a result, managers are constantly faced with trade-off decisions among time, functionality, and quality. By providing a holistic understanding of the risk of release, continuous testing helps to optimize the business outcome.

Reevaluating the Cost of Quality

A critical motivator for the evolution toward continuous testing is that business expectations about the speed and reliability of software releases have changed dramatically—largely because software has morphed from a business process enabler into a competitive differentiator.

For example, APIs represent componentized pieces of software functionality that developers can either consume or write themselves. In a recent Parasoft survey about API adoption, more than 80 percent of respondents said they have stopped using an API because it was "too buggy." Moreover, when we asked the same respondents if they would ever consider using that API again, 97 percent said no. With switching costs associated with software like an API at an all-time low, software quality matters more than ever.

Another example is mobile check deposit applications. In 2011, top banks were racing to provide this must-have feature. By 2012, mobile check deposit became the leading driver for bank selection. Getting a secure, reliable application to market was suddenly business-critical. With low switching costs associated with online banking, financial institutions unable to innovate were threatened with customer defection.

This sea change associated with the quality expectations of software is also reflected in the manner in which software failures are reported. Today, software failures are highlighted in news headlines as organizational failings with deep-rooted impacts on C-level executives and stock prices. Parasoft analyzed the most notable software failures in 2012 and 2013; each incident initiated an average 3.35 percent decline in stock price, which equates to an average of a $2.15 billion loss of market capitalization. This is a tremendous loss of shareholder value. Additionally, looking at organizations that endured multiple newsworthy software failures, it is clear that the market punishes repeat offenders even more aggressively. Repeat offenders suffered an average 5.68 percent decline in stock price, which equates to an average of a $2.65 billion loss of market capitalization. (1)

The bottom line is that we must reevaluate the cost of quality for our organizations and individual projects. If your cost of quality assessment exposes a gap in your quality process, it's a sign that now is the time to reassess your organization's culture as it relates to building and testing software. In most organizations, quality software is clearly the intention, yet the culture of the organization yields trade-off decisions that significantly increase the risk of exposing faulty software to the market.