Agile EVM: Earned Value Management the Agile Way

As an Agile team member, are you trying to figure out where you are, exactly, in the bigger picture of product development compared to where you are expected to be? Are you are working on a government contract where Earned Value Management (EVM) implementation is mandatory, but would like to use Agile software development methods? Perhaps you're an Agile project manager often "translating" Agile iteration results for non-Agile stakeholders? As a product manager, would you like to be able to forecast the impacts of changes on your expected return on investment? Let me introduce you to Agile Earned Value Management (AgileEVM).

This technique offers a simple, lightweight method for integrating the benefits of traditional EVM metrics into your Agile software development process without causing "drag" on team productivity. AgileEVM metrics can provide added value for project stakeholders, product and project managers, and Agile team members. In particular, AgileEVM adds a cost reporting component to the Agile toolkit that supports project decision making.

AgileEVM was born out of my own desire to accurately forecast expected costs at the completion of a fast moving Scrum project with uncertain requirements. As the ScrumMaster and Project Manager on a Scrum project where backlog items were being added, removed, or changed on a nearly daily basis, I was frustrated trying to forecast our expected final costs given the volatility of the project. It was during my ScrumMaster certification class that I asked the question "Has anyone used EVM or EVM-like metrics on Scrum projects?" The question was met with astonishment. "Why," my fellow classmates asked, "would you want to do that? EVM is waterfall. It's not Agile!"

Defining AgileEVM
As my frustration increased I began to search the available literature to find someone who had solved this problem. I had an "ah-ha" moment {sidebar id=1} reading a Wikipedia page defining EVM. The author wrote; "Since EVM requires quantification of a project plan, it is often perceived to be inapplicable to discovery-driven or Agile software development projects. ... Thus, the challenge is to create agile or discovery-driven implementations of the EVM principle and not simply to reject the notion of measuring technical performance objectively." [1]

Determined to solve this conundrum I teamed up with Thomas Blackburn, a colleague and fellow ScrumMaster/Project Manager. We had what we felt was an appropriate solution: translating traditional EVM definitions into Agile terms. We were working to empirically prove its accuracy and importance on two projects. Early results were promising. The research effort really took off when I joined SolutionsIQ. Brent Barton, an experienced Scrum Trainer, project manager, and mathematician listened to our theories and reviewed our results.

Convinced that Thomas and I were on the right track, he worked with us to prove mathematically that the AgileEVM methods were at least as accurate as the current Agile burn down methods. The math proof, our research, and our empirical tests all pointed towards the accuracy and effectiveness of the AgileEVM techniques. The three of us wrote about our efforts and presented the results at the Agile 2006 conference.

The significance of the AgileEVM metrics was again demonstrated to be effective on two recent Scrum projects. On one project, an RSS feeds directory we calculated our AgileEVM metrics after the first Sprint of the second phase. The results supported our intuitive knowledge; we had more story points planned than could be developed by the release date. We shared these results with the Product owner and team members.

Together, we ran a number of "what-if" scenarios using the AgileEVM spreadsheet we had developed - exploring the impacts of various alternatives. The results? The Product Owner agreed to drop the story points needed to bring the metrics into balance. The project moved forward and was completed successfully. The early warning that AgileEVM afforded us proved extremely helpful to the team and to the Product Owner.

On a second recent project, this one a web based payroll system, we had been calculating the AgileEVM metrics from the first sprint as the team got started. By the end of the third sprint it became apparent that the team's velocity was stabilizing, and that we again had more story points planned than could be accomplished by the planned release date. With this information, the team and Product Owner were able to negotiate an extension to the release date, allowing the team the appropriate time to develop those features deemed most critical to the first release.

The Case for EVM in Software Development
EVM is a widely recognized branch of Project Management. It is the subject of in-depth study by the College of Performance Management and is included as a standard technique in the "Guide to the Project Management Body of Knowledge (PMBOK)" published periodically by the Project Management Institute [2]. EVM integrates the areas of technical performance, schedule, and actual cost to provide metrics for work actually accomplished. By comparing the earned value (EV) with the planned value (PV), the efficiency of accomplishing work can be accurately measured.

In a 1998 article for Crosstalk magazine titled "Earned Value Management–A Powerful Tool For Software Projects," Quentin Fleming and Joel Koppelman (authors of the seminal work Earned Value Project Management) wrote "The single most important benefit of employing earned value (management) is the cost efficiency readings it provides." [3]Cost efficiency readings measure how effectively the project is using funds; a key component to measuring expected return on investment for a project or product.

The value of forecasting final costs early cannot be over estimated. In the same article, the authors state that "Earned value can provide any project manager with an early warning tool that sends out a signal from as early as the 15 percent completion point on a project. This signal allows the project manager to forecast the final required funds needed to finish the job within a narrow range of values. If the final forecasted results are unacceptable to management, steps can be taken early to alter the final requirements."

Implementing AgileEVM
There is no question that there is still some push back on using EVM with Agile Software projects. Detractors have claimed EVM should not be used on Agile projects due partly to the belief that for EVM to be applied, the entire project scope must be planned in advance, and in detail using a Work Break Down structure. Another belief is that the EVM technique is too "heavy weight" for Agile projects.

For example, the U.S. Department of Defense has identified 32 standards for EVM must be met on certain sized contracts. Try applying that to an Agile software development project! Other objections center on the fact that traditional EVM calculations include estimates of partially completed work. This is in direct contrast to the Agile definition of work as "done or not done." In practice, none of these objections have proven to be obstacles to the effective implementation of AgileEVM.

AgileEVM has proven accurate at integrating EVM metrics into Agile projects. Our experience in applying these techniques center on projects using the Scrum framework; however we believe that these techniques can be equally applied to other Agile projects as well. Some characteristics of AgileEVM are that:

• It is lightweight and cost effective to implement. AgileEVM leverages already existing work metrics, adding little to no additional work to the current processes.
• It is at least as accurate as current Agile forecasting methods such as burn down charts. To review the mathematical proof of this assertion, review our IEEE published research paper at the SolutionsIQ web site.
• It adds value to Agile teams and stakeholders by providing data for decision making.

Change is expected on Agile projects. It is important to remember that metrics generated by AgileEVM are based on what is actual "right now." Once change occurs, a new baseline is established and re-measurement is required. Fortunately, with AgileEVM this is an easy task.

All this said, AgileEVM is not a silver bullet, and is not meant to replace current Agile metrics or tools. It is simply a one tool among others in the Agile toolkit to provide information critical to decision making. And, AgileEVM is not necessarily appropriate for all Agile projects. Projects with extremely short release cycles will not find as much utility from this technique as projects with longer release cycles involving multiple sprints or iterations. As with any technique, apply AgileEVM judiciously.

Another interesting point to note is thatStory points are what we used for estimating backlog items. They can easily be replaced with whatever measure you are using for your estimating purposes. For example; Mike Cohn has written about using Ideal Team days, or T-shirt sizes, others refer to Gummy Bears as ways of estimating relative size of backlog items. [4] It doesn't matter what the measure is as long as it is reasonably consistent and is expressed as a number. Therefore, measures like T-shirt sizes need to be converted to numeric equivalents to be used in the AgileEVM calculations.

Agile EVM Worksheet
As part of defining and testing these metrics, we developed an easy to use worksheet to calculate and track our results. Leveraging release planning and existing Agile metrics, the AgileEVM worksheet has five initial parameters, and four recurring inputs for easy calculation. In order to use this technique, you must have data to establish the initial baseline:

1. Budget at Complete - what's your targeted budget for the release? This can be expressed in either dollars or hours.
2. Iteration Length - How long are each of your iterations or Sprints? AgileEVM assumes that your planned iterations are of the same length.
3. Planned Iterations - How many iterations are you planning to include for this release?
4. Planned Release Story Points - How many Story points have you estimated to be included in the release?
5. Start Date - What date are you starting the first sprint? While this data point is not required to generate the Earned Value and Planned Value data, by including it the AgileEVM worksheet will automatically calculate your baseline schedule, in much the same was as a traditional Work Breakdown Structure.

Once your initial baseline is established, you need to measure progress at defined boundaries. These boundaries can be developed from Sprints or iterations, or can occur in a time based manner i.e. weeks or months. At each measurement boundary, four simple data points are entered into the spreadsheet:

• Current iteration number.
• Number of story points actually completed.
• Number of story points added to or removed from the release.
• Actual Cost in dollars or hours. It is critical that the actual cost amount used reflects the cost needed to generate the completed story points.

While not a silver bullet, AgileEVM can provide solid data for decision making that benefits team members, product owners, project managers and agile stakeholders alike. 

About the Author
Tamara Sulaiman is a Team Lead, PMP and Certified Scrum Trainer (CST) at SolutionsIQ. Tamara brings over 15 years of experience in business management and software development to a spectrum of industries including: information technology, construction, international development, and education. Since 2003 Tamara has assisted teams in transitioning to Agile methods. As a Team Leader, Scrum Trainer, and Project Manager, she is focused on providing value to key stakeholders and customers through effective delivery of software. As an educator and coach she brings her wide ranging professional expertise to training and mentoring teams new to Agile and the Scrum framework. Tamara is co-author of the original research paper "AgileEVM - Earned Value Management" and has presented her work at the Agile 2006 International Conference in Minneapolis and the Agile Business Conference 2006 in London. She is co-originator of the AgileEVM materials and processes that integrate the traditional project management practice of Earned Value Management with the Scrum framework.

[1] See http://en.wikipedia.org/wiki/Earned_value_management
[2] PMIBookStore
[3] www.stsc.hill.af.mil/crossTalk/frames.asp?uri=1998/07/value.asp 
[4] Cohn, Mike "Agile Estimating and Planning" Prentice-Hall, Nov 1, 2005