Keeping Your Build Under Ten Minutes

One of the practices recommended by Extreme Programming (XP) is to keep a ten-minute build.  Kent Beck and Cynthia Andres write in Extreme Programming Explained (Second Edition): "Automatically build the whole system and run all of the tests in ten minutes.  A build that takes longer than ten minutes will be used much less often, missing the opportunity for feedback."



So what do you do when your build takes longer than ten minutes? 



James Shore and Shane Warden offer some advice in The Art of Agile Development: "For most teams, their tests are the source of a slow build."  Frequently these tests are integration tests -- tests that confirm that your code interacts correctly with third-party dependencies such as databases, files, and web frameworks.  These tests are useful to automate and add value to a project, but it's easy to write too many.



"You shouldn't need many integration tests," Shore and Warden explain.  "The best integration tests have a narrow focus: each checks just one aspect of your program's ability to talk to the outside world.  The number of focused integration tests in your test suite should be proportional to the types of external interactions your program has, not the overall size of the program.  (In contrast, the number of unit tests you have is proportional to the overall size of the program.)"



One strategy our team has used is to replace slow integration tests with fast unit tests.  This means finding the slow tests, deciding if they are testing too much at once (for example, database interaction, web framework usage, etc.), and replacing them with focused unit tests.  We typically do these as slack tasks in our iteration.  But it can be hard work; it often requires reasoning about what the tests are testing, why they were written as integration tests in the first place, and what we need to do to be confident that a unit test will provide sufficient coverage.  Most of the time we find that these tests really didn't need to be integration tests.



However, sometimes projects live long enough (we hope!) that they grow beyond their original scope, and a ten-minute build becomes a challenge.  (I can imagine that Microsoft Word cannot be built and tested in ten minutes, but someone please correct me if I'm wrong!) What do you do then?  Splitting the code into discrete, independent modules that can be built and tested in isolation is one approach.  Another strategy would be to use the power of distributed build servers to run your tests in parallel on multiple machines, but I've not yet tried this (and can imagine many difficulties in setting this up - again, someone please prove me wrong).



How have you practiced keeping your build time in check?