I remember the idealized circumstances of the early years of agile development. I had a small co-located cross-functional passionate team that was using advanced Smalltalk development approaches to develop an even more advanced RAD environment. There was no pre-existing model for the kind of tool we wanted to build. And domain expertise rested with a very few object-oriented gurus. But, we had executive support and were given everything we needed to succeed. Our project was critical to the company's success and we had to minimize time to market. It was a dream job for us caffeine-driven code-slinging techno-junkies.
The Times, They are a Changin'
Agile adoption is rapidly growing. It is being driven from the top-down in organizations that immediately recognize agile's benefits. In others, agile grows more virally and its acceptance becomes an eventuality. As it works its way up, across, and down through enterprises, the stakes get higher. Agile's success and failure now directly influences the success or failure of the business. {sidebar id=1} And guess what guys and gals, with this success comes responsibility. And responsibility these days, especially in larger companies, means reconciling our efficient agile methods with at times heavy-handed top-down management practices. Scary things like budgets, status reports, and governance are part of our daily lives.
Budgets really are a bummer. They constrain a team's velocity to deliver new products and enhancements. But budgets are a business reality faced by all organizations. Management is a bummer too. Given constrained resources, management is always pressuring the team to optimize its throughput. It's not enough to use business goals to prioritize and burn down the backlog. They want more value with the same or fewer resources sooner than planned. And, oh by the way, quality has got to be high as well. (It's really fun to talk about myself in the second person. I've been part of this quot;managementquot; bummer for quite some time now.) Another issue is the difficulty around optimizing a single team's velocity. Optimizing the velocity of a global application development organization is damn near impossible. Especially when you consider some organizations are coordinating thousands of projects and enhancement requests across thousands of resources for thousands of end users. So much for index cards...
In a perfect world (well, at least in my perfect world), everything and everyone would be agile. We'd have homogeneity of development methods across the organization. Everyone would be co-located and our cross-functional teams would purr with harmonious efficiency. Our developers would thrive in their extremely efficient bug-free development environments creating new products and enhancing others. And they'd build their unit tests before the features are even conceived! Well, so much for my utopian dream!
As we all know, large enterprises have a diversity of products, tools, processes and people. Adoption of agile methods necessarily comes incrementally. There is no silver bullet for transforming these legacy products and process. It will be evolutionary, and at times painful, to transition these to more agile development methods. In fact the value should be very closely considered. In the long run, the better business decision may be to encapsulate legacy products and processes and apply agility to new products and enhancements.
To summarize, wide-scale adoption of agile methods across the enterprise necessitates process and tools in order for individuals to interact. The specific challenges include, but are not limited to:
- Managing resources and budgets
- Tracking deferred customer and market requirements
- Providing visibility and governance
- Accommodating process heterogeneity
Managing Resources and Budgets
For decades, software projects, and for that matter all kinds of projects, have been managed to the ever popular triad of schedule, scope, and budget. Unfortunately for customers, some companies also include quality in this list. Let's for the sake of argument assume that quality is fixed. Also, I'll assume that an organization will make the appropriate decision with regard to schedule. Either it's fixed, or a Scrum-like approach is applied and a decision to ship is made when appropriate business value has been delivered. So that leaves us with budget which includes resources and resources are mostly people.
In large software organizations, efficient management of resources is critical. To deliver the most value, we've got to make sure we're always working on the most important stuff, figure out how to get more efficiency out of existing resources, and figure out how to get more resources for a fixed budget (outsource!). The cold hard business reality is that well-managed organizations will pursue all of these avenues. Resource management, a core capability of Project and Portfolio Management (PPM) tools, is becoming a best practice in large software development organizations. PPM tools provide resource management capabilities that allow organizations to understand and manage to their resource capacity, ensure they are maintaining the appropriate balance of resource skills and experience, and efficiently allocate resources and teams to new projects and enhancement requests. Again, this is way overkill for smaller teams, but it's nearly impossible without tool support in large global enterprises that have resources separated by both time and distance.
Tracking Deferred Customer and Market Requirements
Resource management must also be accompanied by efficient backlog management. To make good resource and team allocation decisions, you must have a clear understanding of current and upcoming backlog items. With small co-located teams this is simple. Index cards and whiteboards are effective tools for the team leads (e.g. ScrumMaster, Product Owner) to assign backlog items to individuals. But what do we do when the team is geographically distributed? How do we collaborate with team members in India, China or the Ukraine? I wouldn't recommend faxing or overnighting the index cards. Obvious choices here are issue and requirements management systems. These systems not only help to organize and prioritize the backlog, but also can handle the assignment and status tracking of issues, requirements and user stories. And, as scoping decisions are made during fast and furious development cycles, these systems will help retain information for future release planning.
Providing Visibility and Governance
The object model of application development processes and data is an academic's dream. We can relate the earliest conception of a software requirement down to a defect discovered in a line of code five years later. It's hard to refute the value here until you start to understand the process overhead this implies. And, this is where we have to achieve a balance with top-down compliance-driven visibility and governance initiatives and application development agility.
Capturing key decisions and maintaining relationships amongst the key development artifacts is critical to the business. But this needs to be done in a manner that is transparent to the development team. If it gets in a developer's way of writing code, it will be resisted. Today's application lifecycle management (ALM) environments capture and manage the critical development artifacts and metrics. As these tools mature, they are leveraging the interconnected model and providing automated refactoring and impact analysis support. Further, through integration with PPM and reporting systems, visibility into key metrics and decisions is automatically presented to business stakeholders.
Accommodating Process Heterogeneity
Simplistic models for managing the backlog also don't work for large scale application development where hundreds of applications and/or products may be under development or in various stages of production deployment. In these organizations, backlog items are constantly being created and modified. There are likely to be several thousand items across the products. Some are low-level details and are not relevant to the business. But others are quot;roadmap worthyquot; and can require visibility at the corporate level. So now, when a team is in the heat of battle driving towards a release that's critical to the overall business, and the schedule is getting squeezed, how can team members make a scoping decision and know that they are not impacting the business? And how can this change or request for change be communicated to the stakeholders? This is a development methodology-agnostic challenge. You've got to know what you are working on, who it's for, and when it's required. And you better make sure that if it is business critical, the decision to scope manage a feature out of the release is well vetted. Requirements and issue management tools have effectively been applied to solve this class of problems for years.
Agile methods are no longer relegated to small teams of nonconformists disillusioned by Gantt charts and voluminous specifications. Their ability to predictably create high-quality products that meet customer needs has been recognized and embraced by many large companies. But, to ensure continued success and adoption, we need to figure out how to coordinate our tie-dye shirts with new blue pin-striped suits. The reality of large complex businesses today is that team collaboration necessitates process and tools. Enabling agile methods in large organizations will require creating a fine balance of top-down systems while preserving and encouraging team flexibility and innovation. And, it will require careful and pragmatic acceptance of tools and processes by the agile proponents and trust and support from business leaders.
About the Author
John Scumniotales is Vice President of Product Management for Serena Software. Prior to Serena, he was Vice President of Products for Pacific Edge Software, a leader in Project and Portfolio Management. He was the first ScrumMaster and co-created Scrum with Jeff Sutherland at Easel Corporation in the early 1990's. John has held senior technical and management positions at Rational Software (now IBM) and several other large and small software companies.
