Agile development is a process methodology where, at its core, is the ability for developers to share and test changes in software as rapidly as possible in an efficient but disciplined approach. The Continuous Build process is based on manual scripting and therefore is highly redundant and non-adaptive. Continuous Builds that rely on redundant scripts become road blocks for agile teams when attempting to move quickly from code to deployment. Agile developers must recognize the problems of manual scripting and begin moving toward a non-redundant Community Developed approach to managing the Continuous Build.
Development tools have become highly efficient. From the introduction of IDEs and Service Oriented Architecture, developers have the tools they need to build upon Agile methodologies and improve the overall development process. Source code is generated and reused across teams. Redundant tasks are identified and removed. This is not the case with the build process. It remains a redundant manual scripting process that can cause big problems for teams attempting to streamline their development process from development to deployment.
Rigid Manual Builds
The basic description of Agile development simply does not apply to manually scripted builds. Agile development implies an iterative development methodology. {sidebar id=1} The basic concept is that you build and deploy applications quickly and on a frequent basis. These mini, iterative releases allow for each developer's coding change to be compiled into a complete application on a continuous basis, called a quot;Continuous Build.quot;
The static Continuous Build is problematic for agile developers because agile development results in an application that is changing at a rapid pace. This quickly changing application development process forces updates in the build itself. The agile development process stops while the build scripts are manually updated. The problem is clear. The agile development process is highly adaptive and dynamic. However, the Continuous Build, relies on a non adaptive manual process.
Build scripts are executable documents. Because they are manually written there is redundancy within the scripts themselves. Redundancies are found at multiple layers. Each developer may maintain a local script as well as a shared script that executes the Continuous Build, QA Build and Production Build (see Figure 1). In addition, scripts may be copied so that one set executes with quot;debugquot; turned on and another with quot;debugquot; turned off. Because of the static nature of build scripts, this type of redundancy is unpreventable.
Agile development is highly adaptive but build scripting is not. The Continuous Build relies on non-adaptive manual scripts causing the agile process to be less iterative. A good example of this is the inability for most manual scripts to perform accurate quot;incremental builds.quot; An incremental build is a build that is intelligent enough to recompile only the objects that are out of date, versus the entire application. If the entire application takes one hour to compile, then there can only be a limited number of builds completed in a day. To solve this problem developers attempt to build the application over several machines performing a quot;distributedquot; build. But the problem is not the speed of the build, it is the script's inability to simply re-build the one or two modules that need to be updated.
The non-adaptive nature of build script is out of sync with the highly adaptive natures of development IDEs. For example, code and package re-factoring can cause names of object to change. A developer has the option to easily re-factor code inside of the development IDE such as Eclipse. However, the script itself is not aware of these name changes. The scripts and have no connection to the IDE. What is done dynamically in the IDE must be done manually outside the IDE. Code and package re-factoring tasks can require developers to re-visit multiple scripts to finish the job.
A Build Paradigm Shift
Agile developers are beginning to recognize the high cost of redundant, non-adaptive build scripts. Developers who are successful in defining and working within an Agile development process understand the importance of the build step. Most importantly, they understand the importance of a centralized, Community Developed build process.
There are three primary characteristics of a Community Developed build process: enforcement of build best practices, reusability, and transparency (self documented). The Community Developed build must be able to actively understand when a change has occurred, where the shared source code resides, and how to incrementally build the new changes as efficiently as possible (see Figure 2).
Defining a build best practice is critical to the agile developer. This involves simple steps such as identifying directories where shared objects are to be referenced. Definitions of environment variables should be well communicated such as the CLASSPATH definition. File and directory names should be discussed and agreed upon minimizing the need for code and package re-factoring. The Community Developed build should enforce the use of the same version of the JDK. This will prevent unexpected compiler errors that may be hard to understand. Make sure the system path has the java and javac programs coming from the same location on the file system and that the CLASSPATH used is from the same instance of the JDK. A Community Developed build process allows input from all developers to determine the overall build best practice before anyone starts writing scripts.
Script reuse is critical. Removing redundancies in scripting is the fastest way to improve your overall builds. For developers using IDEs, the build scripts themselves should be based upon the IDE project file. This can be challenging as the builds may need to use quot;headlessquot; mode techniques to build inside the IDE the same as outside the IDE. Agree on how the scripts are to be written. Write simple, transparent, portable and organized scripts. Try to minimize the number of lines of scripting and minimize the number of properties for each builder to set.
Your Community Developed build process should be transparent. Every script should follow the same general logic allowing anyone to understand the process. There should be no hidden or hard-coded directories. This makes debugging a build simply impractical. In order to achieve the incorporation of local build directories into the Agile development effort, environment variable settings can be used and passed to the local build to determine the order in which code should be found and compiled. If the order can be managed, the code managed in the version control tool can be passed to the compiler first, and then secondly the items in the local build directory would be passed to the compiler. It is important to note that for this to work, common components should not be managed inside of the local Project. Common components should always be found by the compiler in standardized directories.
Continuous Agile Builds Must Be Community Developed
A disciplined and well managed build process is the core of any Agile Development effort. Without a well managed central build, there is an immediate breakdown in the process, impeding the creation of executables even though source code may be clearly shared via a Version Control tool. When the build breaks, so does the Agile development process. Agile development can only be a dream if the build is at best a chaotic process.
Builds can be the most chaotic process in the overall Agile development effort requiring attention to detail so that the many pieces fit together correctly. For that reason, the builds must be carefully managed, centralized, and consistent whether the build is a local build or a shared build running on a specialized build machine. If Agile development is to be achieved, the build process must be controlled and repeatable by all members of the development team at all levels, local and shared.
About the Author
Ms. Ragan has had extensive experience in the development and implementation of business applications. She began her consulting career in 1989 consulting to Fortune 500 organizations in the areas of testing, configuration management and build management. It was during her consulting experiences that Ms. Ragan recognized the lack of build management procedures for the distributed platform that had long been considered standard on the mainframe. In the four years leading to the creation of OpenMake Software, she worked with development teams in implementing a team-centric standardized build management process. Ms. Ragan serves as a board member of the Eclipse Foundation and is the COO of OpenMake Software. Ms. Ragan received her Bachelor of Science Degree in Business Administration from California Polytechnic University, Pomona, CA.
