With today's economic pressures coupled with a highly competitive business environment, management is aggressively pursuing ways to increase effectiveness and efficiencies at the same time as they strive to improve customer services. For these reasons many organizations are trying to integrate offshore development into the Agile projects. Offshore development has seen tremendous growth in recent years. The efficiencies gained by combining these two methods could be significant, but there are some pot-holes on the road to success.
Distributed communications, time zone differences, language and cultural barriers, security problems, lack of direct interaction with the business subject matter experts are just a few of the challenges faced when combining Agile and offshore development. All of these challenges are significant and require directed effort if they are to be overcome.
Most off-shore development organization have standardized on the traditional waterfall development as their methodology. These more traditional methods define tasks that have to be done on a large scale, classically organized project. These traditional development techniques tend to run contrary to the fundamental concept of Agile. In Agile there is always a working product which is incrementally improved at the direction of the customer as the learnings are extracted during the demonstrations. Traditional methods set hard and fast requirements upfront.
The Hybrid Model
Many organizations have tried combining these techniques with varying degrees of success. The following graphic illustrates one approach that has worked well in the past. By no means is this meant to say that's the only way you can be successful with and Agile/Offshore - Agile/Waterfall approach, but it is a very common way found in today's environment. That being said the success using this approach still rests in cooperation and flexibility among the onshore and offshore teams.
The Agile initiation process remains the same and a project baseline is established. The baseline defines the parameters of the project, standards, technology architecture and other fundamentals that each interaction will be built upon. Often time this work takes place in what has been called Sprint 0.
In this hybrid model Sprint 1 is the first real iteration in the project. The onshore team works directly with the business user/subject matter experts to define functional requirements. Once these function requirements are in sufficient detail, they are entered in the function back-log. The function backlog is an inventory of capabilities that must be designed, developed, tested, and accepted by the key stakeholder(s).
The Problem
The offshore development team looks at this inventory, evaluates the requirements, size, complexity and effort required for each component in the inventory. Based on these estimates and the size of the development team the onshore and offshore team define and then refine what functionality will be delivered in the Sprint. Once the definition of what will be delivered is agreed upon, the development process begins. This is where the problems start. Most offshore development houses use rigid methodologies. Even the ones that claim to use iterative development approaches like Agile really don't. They are driven by a Sprint task list and want every aspect of the software locked down tight. As the Agile methods progress through what has been called an iterative discovery process using demos and proof of concepts, new requirements are uncovered. This becomes very costly, as billable hours are spent rewriting code and estimating the newly discovered requirements and changing schedules.
But it gets better. Many hybrid (Agile/Traditional) projects use time-box management techniques and call it Agile. Time-boxing is a technique that fixes the time spent on a given task or set of tasks. Once the time-box is established, the development staff does the best that they can to stay within that time frame. So instead working on something until it is completed, as is the case with typical waterfall development, the offshore team only works on it for say, 5 days. At the end of those 5 days the work is either completed or is placed in the backlog inventory to be addressed at a later date, or replaces other functionality that was originally scheduled in the Sprint. The reason this is done, is that the costs have now changed to implement this piece of functionality and needs to be re-evaluated by the onshore team.
Example 1: In one specific instance, the offshore team increased the hours estimate about 1/4 of the way thru the Sprint. When asked for an explanation they said, "they did not include any time in the estimates for proof-of-concepts, prototypes, demos or code reviews." Isn't AGILE all about proof-of-concepts, prototypes, demos? This just proves the offshore team really did not understand the Agile approach development.
The waterfall development process begins with each requirement broken down into functions. Each piece of functionality treated as a line item on the Sprint task lists. As the software is coded and becomes functional, it is walked through with the onshore team, business user/SME as well as the key stakeholder. This is where the offshore development team gets feedback thus clarifying capabilities that were "sort-of" delivered. The customer's comments are integrated back into the design or deferred, and are addressed later. If they comments and clarification are integrated back into the design, the rework is estimated, change orders issued and the cash register rings again.
Another hybrid technique replaces traditional reviews with functional demonstrations. As the work progresses, feedback from the onshore team, business user/SME as well as the key stakeholder is received the feedback is integrated or if it requires significant additional effort it is deferred and entered into the function backlog inventory and addressed in a later Sprint. If the feedback is required at that specific time it is, you guessed it, added to the Sprint task list, the work is estimated, change orders issued and the cash register rings yet again.
As each item on the Sprint task list goes through the waterfall development process, the output is demonstrated and signed-off on by the key stakeholder. At that point of the Sprint, the code based is integrated with other code produced during that Sprint. Once the time allotted for the Sprint development effort has elapsed, all completed items are assembled into one code base. At that point a formal Sprint review is conducted. A formal Sprint review signifies the end of a specific Sprint. The review is a step-by-step/functional piece-by-functional piece demonstration and walk-thru with the business users and key stakeholder. During the Sprint review the delivery is assessed against the sprint goals and objects determined during Sprint planning. Ideally the team has completed the selected functions for that Sprint, but it is not always the case. Since the Sprint review is the close of that iteration, official sign-off also takes place. Implementing this formal review and sign-off process reinforces the wrap-it-up habit and avoids the hanging Sprint problem.
Example 2: In another specific instance, the offshore team basically refused to begin development until they had a formal specification document and functional test cases signed- off on by the customer. In Agile, customer acceptance and approval to proceed is at every demonstration and functional walkthroughs. Again the offshore team reverting back to traditional methods and failing to grasp the Agile approach.
A hanging Sprint is defined as an iteration that is 90% or above done but never completed. The unfinished Sprints have been known to hang around for months. The development team just doesn't seem to have time to go back and complete the work in that they have moved on to the next Sprint.
A Fact of Life!
Many organizations have turned to outsourcing software development projects in an effort to reduce their costs. Despite its benefits, outsourcing carries some unique risks. Forcing an offshore team to adopt and use a methodology that they are not familiar with increases the risk on the project. In addition the fundamental and operational differences between Agile and Waterfall methodologies can and do create substantial cultural change requirements that can not be adopted in a short period of time.
The reduced structure and documentation of Agile create another risk. This risk manifests itself in misunderstandings between the onshore requirements gathering team and the offshore development team as well as between the onshore requirements gathering team and the client's subject matter experts. In addition, where the application area has regulatory compliance provisions the reduced structure and documentation can create significant challenges. Often regulatory requirements mandate process documentation for audits and compliance reviews.
A significant risk that is intensified when using the Agile development approach is in the area of scope creep. Many times the user has difficult envisioning what is achievable while defining a new system. Once they begin to experience the system through the iterative development demonstrations, the "tweaking" and "wouldn't it be nice if we could" requirements expansion begins to occur. The best way to handle this issue is to push off the "tweaking" and "wouldn't it be nice if we could" to future Sprints and estimate the costs associated with them at a future time. That being said, this does tend to cause increased work for both the onshore and offshore teams and drive the overall cost of the project higher.
Perhaps the biggest area of risk is based on the degree of interactivity of software developed in early sprints and the software scheduled for development in later sprints. Highly integrated application can become very inefficient if the requirements for one area of code change through various iterations, the same programming may need to be modified several times over. Whereas, a more formal and complete program specification practices at the beginning, a single area of code could include much more if not all the needs of later programs and be developed with minimal modifications. It basically provides a bigger picture.
Conclusion
There is no question that integration of offshore development into an Agile program can enhance the value to customers by reducing overall costs. However, many offshore organizations have their traditional methodology engrained so deeply into their operations, it is next to impossible to get them to change. Offshore development organizations tend to focus on hard requirement established upfront, but in Agile requirements evolve over time and change. Agile and the use of offshore development that leverages the waterfall development is a tricky, risky, and sometimes very costly venture. While combining the two different techniques is a bit more risky, it is far less risky that forcing the offshore development team to rapidly adopt a new methodology for development that changes their way of operating. If the two parties are just beginning their business relationship, mutual trust has not been established and that is critical when combining Agile and tradition methods. Organizations who decide to try this hybrid development model should first fully investigate and verify the offshore company's true Agile experience and capabilities. For those who are considering the hybrid model, go into it with your eyes open. Combining the two different development styles and approach create substantial risks. You must consider those risks and weigh them against the potential time and dollar savings when making your decision. Software development is tough - this hybrid model makes it tougher.
About the Author
Kevin G. Coleman is a seasoned management consultant with nearly two decades of experience. He brings with him a unique perspective on management, technology, and the global risk environment. He was the Chief Strategist for Netscape and has worked for leading consulting organizations such as Deloitte Touche and CSC Consulting. During his career he has consulted in dozens countries and thirty plus states. Additionally, he has personally briefed fifteen executives from the Global 100 and nearly 100 CEO's worldwide. In addition, he has testified before Congress on technology policies.
