In a recent Dilbert comic strip, Pointy Haired Boss (PHB) tells Dilbert and Wally they will start using agile programming. He explains:
"[Agile programming] means no more planning and no more documentation. Just start writing code and complaining."
All of us who have made the transition from waterfall, RUP, or other gated process to agile can chuckle at this definition. As few as five years back, agile was often seen as a ‘license to hack' rather than a disciplined approach to software development. We've come a long way, but some perceptions persist. One implicit assumption made by Dilbert's PHB is that ‘going agile' is all about programming. He couldn't be more wrong. Moving to agile has effects that ripple through the entire organization, from sales to IT Operations. Some of these effects can be managed through good training. Some are more political than technical in nature, and will require, dare we say it, the delicate hand of your PHB.
What happens when teams adopt agile techniques? How will the rest of the organization react? Maybe you work in a small or idealistic organization with little or no political back-biting or infighting. If so, you are to be envied. Most of us don't live in a perfect world. We are human, and even the best of businesses have a bit of dysfunction here and there. Internal politics, infighting and turf battles will emerge. Your PHB needs to understand and be prepared for this eventuality. PHB can run, but he can't hide from the ripple effects of going agile.
There isn't enough space here to list every organizational pitfall. A lot of them can be managed through education, expectation setting, and the application of basic political know-how. Some, such as changes in funding models, interactions with IT Operations and the clash over availability of the product owner have been documented in previous journal articles. But one issue, the side effects of pull versus push project management, is less well documented but just as potentially damaging. While there are many side-effects of pull versus push, the two I want to discuss are the lack of a master project plan, and reporting status in a mixed-methodology environment.
In waterfall or other gated processes, Project Management (PM) publishes a project plan based on exhaustive up-front analysis and design. PM then ‘pushes' the tasks to the project team. "Here's the tasks, now get to it." We know agile teams don't work that way. There is no master project plan detailing all the work because planning is done just-in-time rather than up-front. The team ‘pulls' work from the backlog for each iteration. And while you might have a master iteration plan, it will be at a much higher level of abstraction than the plan created by the waterfall team.
Why does this matter?
You may not know it, but other parts of your organization use the project plan as a sales tool. If your software is externally facing, you can bet sales will schedule demos to prospects in order to keep sales on the table. Marketing will plan analyst briefings. The executive team will take the plan on the road to talk to important customers. Even if your software is strictly for internal use, your customers will schedule events and executives will make tactical decisions based on your project plan, despite how much it resembles a work of fiction. When you move from a push model to a pull model you take away this important tool. In a pure agile environment, you will never know when, or even whether a specific use case will be satisfied. When sales, marketing or someone from the executive team starts asking for the availability date of a specific feature, the accurate answer is a shrug. Too bad that's not going to be an acceptable answer.
What's to be done?
PHB should engage organizational leadership up-front to set expectations that there will not be a master project plan. He's got to let them know they are going to change the way they make plans, engage customers and deal with analysts. If you can afford it, have PHB bring in an agile consultant and have them run a ‘get prepared for agile' workshop for the executive team. If you can't afford a consultant, have everyone read the Gartner "fact or fiction" report on agile software development.[i] It does a reasonable job of explaining that agile software development changes the entire business, not just IT.
Unfortunately, these your PHB's arguments might fail and you could end up being asked to do up-front planning.[ii] Yes, this is wasteful, but the political reality may require you to start out with some waste, and lean out as your organization gets more agile experience. Have your PHB negotiate again next project, and the project after that, until you come to a workable compromise. As Brent Barton from Solutions IQ says, "Agile is a journey."
Let's assume PHB gets the sales and marketing teams on board. What other problems could you face? If you are in an agile-only environment, the next issue probably won't be troublesome, but if you are in a mixed methodology environment, look out. You have to face The PMO project status meeting. Here's problem: On a Waterfall team, because your tasks are pushed to you, you report on how closely you adhere to a published plan. On an Agile team, because you pull tasks, you report how much is left to do, and how much you can do. About halfway through the project, when waterfall projects start to get derailed, these differences will become political.
When you are on a waterfall team and you fall behind, you generally go through a re-scoping effort with lots of gates and visibility across the business. If you have to re-scope several times, this on-going negative visibility will do your team no good. Your PHB may start getting defensive and sniping at the PHB of the agile team. "His team is cavalier about requirements. When they get behind they just throw stuff into the backlog. There is no discipline! There is no accountability!"
He's right, of course. When you are on an agile team and you have too much work, you de-scope as part of the agile process. You don't need to execute a change document. You don't need to get permission. The rest of the business may not even realize that your project scope has changed. At project status time your agile team looks forward to what can be done rather than backwards to whether or not you have adhered to plan. This causes a lot of friction and backbiting between the project teams. We don't like getting negative feedback. We especially don't like it if we think others are held to a different standard.
To restore good order and discipline, both waterfall and agile teams must report on an apples-to-apples basis. We already know that agile teams can't report against a project plan. Can a waterfall team report work remaining and velocity similar to the agile team?
I believe so. In an agile environment we calculate velocity based on the project's history to predict how much work we can expect to get done over time. We can take a similar approach on waterfall development teams. As each task is completed in a waterfall world, we can calculate the percent overrun. We can average the overruns over the project to date, and use it as a predictor for future tasks. It isn't perfect by any means, and just as with agile velocity, it will need to be revised and refined as the project proceeds, but it will give the waterfall development team the means to determine, not just how far off plan they are at the moment, but how far off they are likely to be at the end. Nobody likes to de-scope or push dates back, but if you're going to give bad news, it's better to give it early and give it once rather than having to give it over and over again.
Now the teams can be on a similar footing. If you are on the agile team you report on what you've accomplished, what is left to do, and how much you can actually get done in the allotted time with the allocated resources. Now, if you are on the waterfall team, you report pretty much the same, with the understanding that you don't have the choice to re-scope at will. It's not a perfect solution, but if your PHB confronts the issue up-front, he is likely to avoid some nasty confrontations later.
What's the take-away for your PHB? People can be mean and petty. Politics and infighting can hurt or even derail a project. Handling touchy issues up-front, to include the ripple-effects of pull versus push project management, will help defuse potentially destructive situations and allow you to roll up your sleeves and get to work, coding and complaining.
About the Author
Dr. Kelly Shaw has been in the software business for 25 years as a developer, development manager, and architect. She currently works as an analyst for Serena Software, researching over-the horizon trends in software development.
[i] Agile Development: Fact or Fiction, Gartner Research, October 2006.
[ii] This approach is also tied up with the problem of funding projects in organizations employing both waterfall and agile techniques. Funding discussions are on the list of "Things Pointy Haired Boss Should Know." Unfortunately there simply is no room in this article to discuss the issue.
