Is Iterative Development Still Relevant?

Iterative development dates back to the 1950s when it was used by NASA in Project Mercury, but it wasn't until the 1990s that it was leveraged in the Rational Unified Process (RUP), a framework that was widely used throughout the industry. I remember being trained in RUP in 1999 and at this point, it was being hailed as the solution to the all-too-common failure of large waterfall projects. Agile and Scrum were almost unknown and Iterative Development was the hot new thing in IT.

What Is RUP?

There are numerous websites on RUP and one of the best sources is a white paper produced by Rational Software (later to become part of IBM). Here I will just offer a brief explanation within the context of Agile.

RUP is divided into four phases: Inception, Elaboration, Construction, and Transition. But as we can see from the diagram below, it is not as rigidly sequential as a traditional Waterfall project. The activities are overlapping.

For example, in a Waterfall project, requirements would often be documented and signed off before the next phase could start. However, here we can see that many of the activities run throughout the project. For example, we can see that the Business Modelling is done mostly upfront during Inception and Elaboration but continues throughout all four phases of the project. Similarly, testing starts in the initial phase and continues throughout the project.

The Problem of Contracts

Organizations often tell me that one of their inhibitors to adopting Agile frameworks is the contracts they have with their clients. In the commercial world of projects, there are, broadly speaking, two types of contracts:

1. Fixed Price: The scope, cost, and timeframe are defined up-front. Payment milestones are often linked to delivery milestones (such as requirements sign-off, code released to UAT, etc.)
2. Time and Materials: The client is hiring a vendor project team and agrees to pay them for the hours they work. Normally it is based on the submission of monthly time sheets.

There are also numerous variations and hybrids.

Clearly, Fixed Price contracts, with their scope, timeframe, and cost locked down, are not very Agile. It is difficult to imagine that the project team would "welcome changing requirements, even late in
development" (Agile Manifesto) under these circumstances.

A Time and Materials contract is more suitable for Agile development as it would likely contain a Rate Card but little information on scope or timeframe. But will clients agree to this type of contract, especially in the four scenarios mentioned above? Many clients want to know what they are going to get, how much it will cost and when they will get it. And fair enough, after all, not many of us would buy, say, a car or a phone without being clear on these things.

One of the values of the Agile Manifesto is "Customer collaboration over contract negotiation." This sounds great and of course, we all want to build a collaborative relationship with our clients and not waste time negotiating every clause of the contracts. But building a collaborative relationship takes time to build up trust. And what if:

1. We are a start-up with very little track-record.
2. We are working with a new client who has no first-hand knowledge of our company.
3. We are working with an existing client and have had a turbulent relationship; trust may have been damaged or strained.
4. We are in a competitive situation with the client assessing detailed proposals from multiple vendors. Saying "Don't worry about the contract, let's just collaborate" may not help us to win the deal.

Even with an internal project, where no formal contract exists, senior management may be unwilling to fund a project until convincing value cases, business cases and ROI have been submitted. These artifacts require a scope, cost, and timeframe to be estimated ahead of the project's start.

How Can RUP Help with Contracts?

In RUP, the Inception phase (which roughly equates to a feasibility study) could be done either for free or with a small stand-alone contract. At the end of Inception, a high-level forecast is delivered containing a vision of the core requirements and a costing forecast. It would be enough to draw up a contract for the remainder of the project (with some in-built variance).

How Agile Is RUP?

What does RUP have to do with Agile? The answer is Incremental Development. All four phases of RUP are broken down into Iterations (roughly mapping to Sprints in Scrum) which, according to the white paper, results in "a release (internal or external) of an executable product, a subset of the final product under development, which grows incrementally from iteration to iteration to become the final system." The wording could almost be lifted from the Scrum Guide. Even the first phase of RUP, Inception, results in the release of a prototype, i.e., working software.

So the benefits of Incremental development, as often stated about Scrum, equally apply to RUP:

1. Short cycles help to focus the team, harness motivation, and develop a sense of achievement.
2. Design errors or deviations from requirements can be spotted and corrected early in the project.
3. The iterations serve as a learning experience for the team and promote continuous improvement.
4. Stakeholders get to see what is being delivered which inspires confidence.
5. Testing can be done throughout the project rather than all at the end, thus cutting the cost of fixing defects and reducing risk.

There are, however, aspects of RUP which are not particularly Agile:

1. There is no mention of self-managing teams or servant-leaders. The traditional project roles (Project Manager, System Analyst, Developer, etc) stay the same.
2. There is no, explicitly stated, concept of Kaizen (continuous improvement) or Retrospectives in RUP. Although the iterative approach should lead to some learning throughout the project.
3. The majority of the scope (but not all) is defined up-front so welcoming new requirements late in the project, could be problematic and would likely require a Change Control process. (However, in certain types of projects, defining scope up-front is doable and even beneficial).
4. There is no mention of face-to-face communication or the colocation of teams (but there would be nothing preventing doing this in a RUP project).

When Would RUP Be a Useful Approach?

I have spoken to several organisations who are keen to go Agile but are limited by constraints such as those mentioned above—demanding clients who are not prepared to risk investing in projects where deliverables, cost, and timeframe are not known up-front. The conversation often turns to some sort of iterative hybrid where some, or all, of the scope, is defined up-front but delivery is incremental.

I believe that RUP is still useful and relevant in certain situations where Agile may not be possible:

1. As a stepping-stone to a more Agile approach providing time to develop trust between client and vendor
2. As a pre-cursor to Agile, where teams and stakeholders can get to see the benefits of incremental development before going fully Agile
3. As an alternative to Agile where requirements are well known and timeframes are fixed (e.g., some compliance projects)

In any case, when discussing projects, contracts, and delivery approaches, I believe it would be wise to keep iterative and incremental development on the table.

References

Iterative and Incremental Development: A Brief History (PDF here)
Rational Unified Process. Best Practices for Software Development Teams (PDF here)