6 Steps to Bridge DevOps with Release Management in the Enterprise

All around us, software development is the engine of innovation. Software innovation disrupts existing business models by enabling products and services that are more intelligent and targeted. To ensure that they are the disruptors and not the disrupted, enterprises invest significant capital in managing the software development lifecycle (SDLC).

However, a gap in the SDLC is throttling back the velocity of new application innovation and causing the failure of critical business systems. This gap sits between development and operations—a traditional “no-man’s land” of responsibility, tooling, and focus. Bridging this gap has to be a top priority for the modern enterprise.

For the highly regulated large enterprise (HRLE) in sectors such as financial services, insurance, health care, aerospace, defense, and government, dealing with this gap is critical. A failed software release can destroy brands, reputations, regulatory standing, and other assets that can take years to rebuild.

Change Management: Necessary but Not Sufficient

Many HRLEs have adopted the ITIL process framework and implemented IT service management applications that include change management modules in order to support their operational processes. Though we often hear the terms change management and release management used interchangeably, they represent two distinct and important disciplines in the SDLC. Change management is about determining what changes are going to be accepted, implemented, and deployed. Release management is about deciding which changes should go together, tracking them through the lifecycle, providing visibility to those changes, and ultimately ensuring that they move safely into production.

Release management is concerned with the movement and implementation of the change request through the application development lifecycle. Generally, it begins when the release gets a name, e.g., 4th-Quarter Release. That’s when content—first the change requests and then the requirements, code, test scripts, test results, sign-offs, etc.—gets assigned to the release and moves from development, to test, to systems integration testing, to user acceptance testing, to preproduction/staging, and then on to production. The practice of release management ensures that the enterprise follows the release process through the entire SDLC and that the right artifacts move from stage to stage completely, safely, and speedily.

Without strong release management, HRLEs often find the management and communication of changes to be bewilderingly complex, especially given the huge volume of changes moving into production every week. By grouping a collection of smaller changes together in a set, release management vastly simplifies the complexity.

DevOps: Beware the Purists

Some organizations are experimenting with DevOps, but it is not the promised panacea for HRLEs.

Pure DevOps preaches the blending of roles and sharing of responsibility between development and operations teams, in some cases creating DevOps units trained across their respective disciplines.

However, in an HRLE there are important reasons to keep development, QA, and operations separate:

• Regulatory compliance requires the segregation of duties between development and operations
• It is beneficial to keep scaled benefits achieved by consolidating the data centers and private clouds under a single, shared, operations team with development aligned to specific business units
  
• Division of labor benefits from experts in operational or technical areas that can be more productive leveraging their areas of focus

While some commentators have stated that DevOps “won’t work” for the large enterprise, it is not that simple; HRLEs can benefit from some DevOps practices, but they need to take particular care to separate what is applicable from what is damaging, costly, or impractical.

Continuous Delivery: Only Half the Answer

Continuous delivery is another popular practice for many organizations. A fundamental building block of continuous delivery is a “CD pipeline,” which automates the movement and progression of developed code across the SDLC into production—building, testing, staging, and promoting. Enterprises build a CD pipeline using tools to automate builds, deployment, and configuration management.

A working CD pipeline goes a long way toward enabling improved release management. Automating steps in the SDLC eliminates manual labor, reduces errors, and improves repeatability.

CD pipelines are very successful in companies built around a single application and a modern, simple infrastructure. However, HRLEs are vastly more complicated, with dozens of software applications, crisscrossed with complex interdependencies, and running in a myriad of old and new environments (including mainframe systems).

For both regulatory compliance reasons and internal visibility, HRLEs need to be able to trace a change in production back to the requirements, defects, or enhancements that triggered that change, along with the corresponding source code updates.

Release Management: Six Steps to Success

If ITIL, DevOps, and continuous delivery initiatives are not enough, how should HRLEs mind the gap between development and operations?

The following six steps will help the HRLE master proven practices of release management and ensure the success of application deliveries.

Step 1: Assign a Leader
The first step is to assign a clear leader. Any potential leader must be seen as a credible and honest broker who will listen to the issues and needs of all groups. The enterprise needs to balance the goals of development and operations, or to “move fast without breaking things.”

If release reports into the development side of the house, the tendency is to be optimistic about the software changes and move them into production quickly, but quality and production stability can suffer as a result. If release reports into the operations side of the house, the tendency is to be more pessimistic and slow the delivery velocity down to allow more testing time, but production quality is higher. Often, in HRLEs, release management is the responsibility of the quality assurance organization, which sits “between” development and operations.

Release leaders must have an understanding of the enterprise’s SDLC, the technical and process aspects of release management, and a visceral understanding of the enterprise’s tolerance for risk. Most importantly, the leader needs support from senior executives and empowerment to drive change in development, QA, and operations practices.

Step 2: Define Objectives and Metrics
Start at the end goal and then work out how to get there. It’s also important to define how to measure progress along the way—clear, measurable metrics help unify the development and operations teams around a common goal.

There are many possible ways to measure release. The following are all valid objectives:

• Increasing release (or change) throughput
• Reducing unscheduled releases
• Improving scope adherence
• Shortening lead time (the time from code commit to production)
• Increasing the quality of the end deliverable
• Improving awareness of release timing, schedule, and milestones

Step 3: Map Out a Disciplined Process
A collaborative release management process, with defined workflows, gates, approvals, and information flow supporting multiple tracks through the release, will ensure accountability and responsibility. Eliminating unnecessary approval steps can improve both the quality and frequency of releases by focusing accountability on fewer people.

Enterprises should plan to spend as much as six to eight weeks of effort on this step. Understanding existing processes and designing better ones is complicated and takes time.

Step 4: Control the Content
This means developing a strategy to make sure that all application artifacts, not just source code, are in version control and that automation is used across as much of the application release process as possible. Inconsistency across environments—development, component testing, system integration testing, user acceptance testing, preproduction, and production—due to uncontrolled assets can cause needless thrashing or, worse, a failure in late stages of deployment.

HRLEs should build a single, hardened source code management system to make sure their software assets are secure and controlled. Too many enterprises allow the proliferation of source code repositories, a situation made worse if based on open-source technologies such as Git and Subversion, designed by developers for developers, without the security and compliance required by the HRLE.

Step 5: Build the Supporting Infrastructure
Along with seeking specific functional criteria, prospective buyers should also ask the following questions about potential vendors:

How much experience do they have supporting the special needs of highly regulated large enterprises? Does the vendor have a history of success in this complex domain?

Do they provide “out-of-the-box” security and compliance with their products, or is it something that needs to be added after the fact? Do the products scale across widely distributed enterprise teams or are they applied mainly to smaller workgroups?

Do they provide an integrated set of products across source code management, process control, and application deployment automation that will work together without extensive customization and maintenance? Do they work with the other tools in our SDLC? Does the vendor help enterprises across mainframe and open systems?

Step 6: Set Up a Governance Approach
A well-functioning change management process including multiple, hierarchical change advisory boards (CABs) is a precondition of good release governance. CABs need accurate information, a controlled process, and a living release calendar.

Change management processes enable real-time sharing of information between operations and release. In enterprises that master the six steps, release management is a well-grooved process with the following attributes:

• Visible: The full status of upcoming software releases is clear to everyone
• Controlled: Each step in the process is enforced and tracked and exceptions are rare
• Compliant: One system of record provides all data needed by auditors and regulators
• Error-free: Software updates perform as required or are immediately recovered
• High-speed: Software gets released in hours and days rather than weeks and months
• Efficient: Nights and weekends are rare and automation is used extensively
• Secure: The process, intellectual property, and application are protected from threats

Mind the Gap

Software development needs to go faster in nearly all enterprises to compete in the modern world. However, the gap between development and operations often defeats this goal. Bridging this gap is a top priority, especially for highly regulated large enterprises.

Addressing release management should be a priority to streamline operations at any large enterprise within a specialized sector. Even if you are already using release management to ensure the success of application deliveries, it is a process of continuous improvement. Wherever you are on the journey to better release management, there is always more to do to mind the gap.