XP: That Dog Don't Hunt

I have some major concerns about eXtreme Programming. XP may be a great way to develop software but, as currently constituted, it's really bad business. XP's fundamental premise that the software will be done when it's done and will cost what it costs is the antithesis of sound business practice. Its insistence that the customer be involved throughout the development process is equally unreasonable. If XP proponents don't successfully address these problems, XP will suffer the same fate as the long list of other iterative practices that preceded it.

To be sure, XP insists on some very good technical practices, among them test-driven development, continuous integration, and the adoption of a single coding standard to be adhered to by all the programmers on a development effort. Other core practices, however, are of questionable merit. In explaining the "Whole Team" practice, for example, Ron Jeffries, one of the founding members of the Agile movement, says on his Web site (XProgramming.com), "The best teams have no specialists, only general contributors with special skills." This seems to me to be good evidence of XP's intent to keep software development in a craftsman stage, in direct opposition to the business objective of seeing it evolve into a more engineering-like discipline.

My primary objection to XP and the other Agile methodologies, however, is the disrespect it shows for management's role in the process. The whole thrust of these methodologies can be summed up with the phrase "you'll just have to trust me." Sorry folks. As we say in Texas, that dog don't hunt. One of the most "in your face" practices is the "one-page requirements document." Someone should let these folks know the Internet bubble burst. The "plan on a napkin" days are over. Really. Another objectionable practice is what they call "the Planning Game." Again, from Jeffries' site, we are told "[t]he emphasis is on steering the project-which is quite straightforward-rather than on exact prediction of what will be needed and how long it will take-which is quite difficult." So XP practitioners aren't interested in working on the hard problems? A new one, to me at least, came out of a session on Agile Testing at the XP/Agile Universe conference. Quoting from the write-up on the ObjectMentor.com Web site, "Use a whiteboard or colored cards instead of a defect tracking system-faster feedback." Feedback to whom? Doesn't management, who's funding this effort, deserve any insight into the process? It seems the Agile movement's position on management's role is "hand over the money and leave us alone." One word comes to my mind: adolescent.

The proponents of XP seem to know full well what the foundation of the problem is: end users cannot easily visualize the solution to their problem. Martin Fowler, another of the Agile movement's founders, says on his Web site (MartinFowler.com), "Only when you use an early version of some software do you really begin to understand what features are valuable and what parts are not." But do they address this problem? Only insofar as it lets them get on with programming. "We'll just do it in little pieces. That will make it easier for the end user to get what they need. But we can't tell you how many pieces are needed to solve the whole problem. So the end user will just have to stick around." Great for the programmers. An open checkbook and an audience too.

To be viable for the mainstream, XP needs to address the needs of the people who pay the bills. We must be able to show the C-level that we know what is needed to solve a business problem before we start building that solution. Those are the rules everybody else plays by. We can sing "software is different" 'til the cows come home. But in the end, the rules don't change, the players do. Luckily, there is hope on the horizon. A new generation of prototyping tools is allowing nonprogrammers to very rapidly produce highly functional prototypes. Functioning prototypes can serve in the place of most of the voluminous requirements documents we've previously used to show management that we knew enough to start coding. Those prototypes also give the developers enough knowledge to predict how many pieces they'll need to build to complete the whole thing, so we'll have the estimates management requires. Now here's a plan that could work. The end-user can see what they're going to get, bang on it to make sure it'll do what they need, and get back to their day job. Management has the estimates and assurances they need, so they're comfortable funding the effort. And the programmers can build the thing the XP way, if that's what they want. Win-win-win.

Why do I care? After all, my position is that XP, if it does not successfully address these fundamental problems, will fail of its own accord. The reason I care, is this: About two million of America's best jobs are on the line. Software development has failed American business repeatedly, consistently. First there were the ERP debacles of the 1990s. Then Y2K. Then Internet mania. Now XP says to the executive, "The problem is, you've been going about this all wrong." I wonder how much more the execs will take before they say, "You know, you're right. We have been going about this all wrong." The Indian, Russian, Chinese, and other outsourcing firms have been saying to these same execs, "We understand what you want, and you can have it. And you can have it for 40 percent less." Maybe XP is just what our foreign competitors have been waiting for.