The term “continuous integration” is getting a bit of attention these days. It refers to the process of integrating often (or immediately) to reduce integration effort, complexity, and pain.It allows for others make changes more readily. While the term “continuous” is catchy, it is not accurate in what the concept implies. In context to integration, it implies a process without interruption.
In software development, while changes are not continuous, this means to integrate often or frequently. Thought should be applied to determine when and how frequently integrations should occur, to re-work the development process to support frequent integration, and to setup an infrastructure that can support this approach. This paper provides some useful guidance in establishing an environment where frequent (or continuous) integration may be possible.
Understanding the Approach
In general, continuous integration is the process when code is checked in and is immediately built in the integration stream to see if it can build with existing code. If the build is successful, then an automated set of tests are run to validate the changes. Once the build and test are successfully completed, the results are made available to the rest of the developers. The premise is that when you continuously integrate, you can continually encounter and overcome small manageable integration issues during development avoiding large time-consuming integration issues.
Consideration should be given to whether a continuous integration approach is right for you. Most projects currently operate in the classical integration approach. This involves changes that are not frequently made; these may be quite large and include changes in numerous files. Other developers do not have to use these changes until they are ready. The continuous integration approach applies the changes to the public very frequently, the changes are typically small and precise (change only what exactly needs to be changed) where one or few files are changed, and the developers must be ready to accept the changes after they have been successfully built and tested.
Effectively, the continuous integration approach is a cultural shift for many developers who, following the classical approach model, like to work securely in their static workspace and not have to consider other changes until they are ready for them. The issue with the classic is that the changes from developers are not applied very frequently, therefore integration (e.g., merging) may become a significant and often painful effort. The continuous integration approach reduces the significant integration effort, but requires a different development methodology whereby developers are working on smaller change requests and change sets which require less time to complete. With all of this in mind, how would a group change from the classical approach to the continuous integration approach?
In order to start applying the continuous integration approach, consideration should be given to what needs to be in place to make this happen. First, there must be a development methodology in place to support this new concept. Second, there must an infrastructure in place that can support the continuous integration approach.
Development Methodology
When considering the continuous integration approach, give thought to the development methodology, as it is much more effective when the work is chunked into smaller tasks. The requirements should be precisely written to represent small changes. The software should be developed in small increments or iterations and testing needs to be integrated into this process.
Other Agile related methodologies include Scrum, Dynamic System Development Method (DSDM), Rational Unified Process (RUP), and Rapid Application Development (RAD).
Development process
The next step is to take the selected methodology and to establish a development process. For example, let’s say that a project manager plans very discrete chunks of work (or change requests) that can be completed within a short period of time as prescribed by the development methodology. This methodology works best when the development tasks are assigned to specific developers targeting specific and non-overlapping functionality to reduce or avoid significant merging activities. When the developer receives a change request, he checks out the very specific pieces of code, makes the change, builds and unit tests, then checks the code into the integration stream. Upon check-in, an integration build immediate occurs that includes testing. If this build and test are successful, the other developers are notified of these changes and they may then synchronize their workspace with these changes.
Cultural Shift
In order to apply continuous integration, an agile-related methodology must be considered and changes to your development processes must be made to support this approach. Given the adversity to change in most organizations, this is not small task. It represents a significant shift in the development and build process, but focuses on small and steady progress. Integration should happen frequently (possibly more than once a day), but this will vary from project to project depending on the nature of the work. It is better to start with a new and small project that is appropriate for use with agile methodologies. This way, there is no existing culture and methodology to overcome and training can be applied that teaches the developers the new approach.
With this in mind, there needs to be an intelligent and conscious effort to select a development methodology that supports small increment or iterative development. Broadly, Agile ideas are best suited for applying continuous integration. Inversely, continuous integration is a key element of many Agile related methodologies. An example of such a methodology is extreme programming (XP). XP defines practice areas that include (but are not limited to) planning for small releases, focusing on only the changes that are necessary, making the change requests small and manageable, building and testing continuously, working in small groups particularly in pairs of developers, and building several times a day so all developers have the same baseline of code.
The next step is to take the methodology selected and establish a development process. An example may be the project manager plans very discrete chunks of work (or change requests) that can be completed within a short period of time as prescribed by the development methodology. This methodology works best when the development tasks are assigned to specific developers targeting specific and non-overlapping functionality to reduce or avoid significant merging activities. When the developer receives a change request, he checks out the very specific pieces of code, makes the change, builds and unit tests, then checks the code into the integration stream. Upon check-in, an integration build immediate occurs that includes testing. If this build and test are successful, the other developers are notified of these changes and they may synchronize their workspace with these changes.
Infrastructure
What needs to be in place to integrate continuously? First, we need a fast and centralized server to integrate, build, and test the code quickly. Second, a solid configuration management (CM) technology is needed. Third, we need a continuous integration technology that supports automated builds and testing. In addition, while having successful builds is important, the automated testing piece is critical so that the changes pass specific test criteria before it is make publicly available. Once these are in place, you can create a continuous integration environment.
· Fast and centralized server: Since continuous integration implies frequent building, it is important to have a fairly fast and powerful server to handle the constant builds. The central server approach is needed so that the continuous integration technology can monitor the CM repository (both technologies needing to reside on the same server) so that they can work effectively together.
· CM technology: It is important to have a quality CM technology in place. In relation to continuous integration, the CM technology needs to be available on a centralized server. Good CM tools to consider (but not limited too) are AccuRev, ClearCase, Perforce, and SerenaVM. There is also CVS, the freeware CM tool.
· Continuous integration technology: The continuous integration technology is needed to monitor the CM repository in real time and then initiate events (e.g., kicks off the build, tests, and reports on the results) when it notices changes to the repository. There are various tools maturing in this space.
Using CruiseControl.Net as an example, this technology tracks the project’s CM repository for changes (based on check-ins), then uses NAnt to do the build. When the build is complete (whether successful or broken), an email notification is sent to the development team. This may then initiate some testing.
Summary
Continuous integration may be helpful for certain projects. It can help reduce the effort, complexity, and pain of large integrations by frequently building code and making it available to others. To begin applying the continuous integration approach, intelligent decision-making and planning must occur in the area of development methodology (to include Methodology options, development process, and cultural shift) and the infrastructure (CM technology, fast and central server, and continuous integration technology). The good news is that this shift in culture may have promise for the future of software development.
References
1. Continuous Integration by Martin Fowler & Matthew Foemmel - http://www.martinfowler.com/articles/continuousIntegration.html
2. The New Methodology by Martin Fowler - http://martinfowler.com/articles/newMethodology.html
3. XP (Extreme Programming) Integrate Often Lessons Learned - http://www.extremeprogramming.org/rules/integrateoften.html
