SCM Design Patterns: Parallel Development and Content Management

This author is not a storng proponent of the Parallel Development, the fifth design pattern. I feel that it has some fundamental flaws that make managing the quality and consistency of your CIs very difficult. Parallel Development issues from my perspective:

• Introduce errors into your Application at points in the development process (usually later States) where they are difficult to find and fix
• When Merging it is inherently difficult to ascertain the impact of a change on the Application as a whole unless stringent testing occurs along with the new merged versions
• Personnel who have made changes off the mainline trunk are not always present when a merge has to occur. Inappropriate personnel (quite often the SCM Administrator) get stuck trying to guess what should be merged or not 
• Text comparators within SCM tools or IDEs do not do a good job of helping the developer understand the full impact of a change in the Application when merging 
• Lack of communication up front in the design process causes developers to execute branching to get around design issues
• Takes more time to complete development due to the added complexity of managing branching and merging
• Introduces a higher level of risk than Single Line Development

Admittedly, there are some legitimate reasons for Parallel Development:

If you do implement a Parallel Development model it is important to establish a strict branching, merging and approval strategy/policy. This is needed so that all stakeholders in the development process know what the branching and merging rules are and expectations of how the Parallel Development model will impact the development of an application.

Figure 14 shows a Parallel Development implementation that I have seen work well. 

This Parallel Development design pattern would be considered simple by many, but I feel it has some key features that make it very reliable, has high quality, and is manageable.

• The branches follow a subset of the mainline branch which incorporates an initial level of quality control
• The branch CIs flow into the Development State of the mainline branch so that all the States will be gone through, thus insuring the high level of quality that all other CIs have
• Since the Branch CIs enter the system in the Development State of the lifecycle they do not impact CI configurations in later States
• There is a dedicated developer who analyzes the branches as they enter the Mainline Branch

The baseline trunk that is indicated in Figure 14 should be periodically refreshed at important points in the development process (version releases, point releases, major new functions) so that the branches are starting off from a recent version of the Application. For those who wish to explore more of the Parallel Development Design Pattern, the following two links will take you to two informative white papers on the topic in CM Crossroads: Streamed Lines: Branching Patterns for Parallel Software Development

ABCs of a Branching and Merging Strategy
Parallel Development requires a sophisticated SCM tool to allow a development team to be successful. Parallel Development also requires both a comprehensive and intuitive user interface as well as a database that can track the various branches and merges effectively. When integrated with a multi-state Design Pattern, we have reached a high level of SCM functionality found in only a few of the SCM tools on the market today.

But there is a unique flip side to this ever-increasing level of function and complexity. The internet has demanded many conceptual changes in the development of applications. Speed of development is always a concern in any software endeavor, but Web development has pushed the pace of development to new levels not seen before. Complex modern web sites have also brought together disparate teams that have not usually had to work together in the past, or are on the other side of the world. The last Design Pattern will deal with these and other issues to support Web Development.

Design Pattern 6: Content Management
The last Design Pattern I would like to discuss is a bit of a niche pattern that deals with Web content item (CI) management. Content Management (CTM) can be viewed as a less structured form of SCM. CTM was born out of the need to have multiple organizations submit CIs into a test web server environment and rapidly see the results of the changes. The topology (see Figure 15) is usually comprised of an organization/company that hosts the Website and outside organizations/companies that need to access a test environment. Instead of a small grain approach of assigning versions to individual CIs, a CTM system works on a large grain concept of the whole site being a version.

This allows for the rapid iteration of CIs that make up the Web site. An issue that this rapid iteration process brings up is that of Security/Risk. The security issues discussed in Design Patterns 3 and 4 are somewhat relaxed for this Design Pattern. This does pose a risk as there is less control but due to the rapid iteration paradigm, errors can be corrected more quickly. Firewalls are usually a method for keeping computer users out, but in this Design Pattern special rules for Firewalls must be configured so that outside organizations can have access to the CTM system, deposit their CIs and see the results.

Since the architecture of Web Applications has become so complex in recent years, many development teams have gone back to a more structured SCM environment as we discussed in Design Patterns 3 and 4. This Design Pattern however works very well for simple to intermediately complex websites that are primarily HTTP (Hypertext Transfer Protocol) based with limited use of more advanced components such as application and database servers.

Following the flow of Figure 15 we see the Development Teams (I use the term loosely here) depositing their CIs into the Test Web Server.

• Content Providers*: organizations that supply graphic/data CIs that form the information that is presented in the web site (not necessarily how it is displayed)
• Sales Staff Information*: organizations that supply data about selling something or request from web site visitors information about buying something
• Design Firm Information*: organizations that are contracted to develop the “look & feel” of a web site. Get involved with the graphic presentation of Content Providers and Sales Staff information. CIs are the graphics (.gifs, .jpg) that make up the site
• Code Providers*: organizations that pull together the first three teams CIs and create the actual web site. This team can also be involved with back end data sources and other non-graphic activities such as web site configuration

*Note: The roles I have created for this Design Pattern are for illustrative purposes only. Your development environment and SCM system/tool may call for other/different roles. These teams can rapidly view and update their CIs until they feel their constituent part of the Web Site is complete and functions correctly. There is usually a “Web Master” who acts as traffic cop to control the makeup of the Test Web server. There is then a cut off date in which all components must be submitted to the Test Web Server. The Web site owner/testers then review the site for completeness/functionality/quality and then decide whether or not to move the CIs that make up the site as a whole to the Deploy Web server. If this group does not approve then the appropriate Development Team fixes their CIs and the Site is reviewed again. Once approved the Site as a whole moves to the Deploy server. This server functions as the repository for the Production Web Server(s). There is also a deployment function of copying the Deploy Web Site as a whole to the Production Web Servers (controlled/executed by the Web Master). If there is a problem with a Production Web Site, it can easily be refreshed by deploying the site from the Deploy Web Server. CTM Tools support this rapid/iterative development style by providing several key functions:

• Graphics based web interface to support design teams involved with the graphic design of the web site
• Large Grain approach to CI management (all CIs in a release are a version)
• Tools for managing the deployment of web sites
• Web site management tools for Web Master

Conclusions
In this whitepaper we have seen six Design Patterns that highlight many functions/features of SCM systems. 

• Design Pattern 1: Paper Forms SCM - determining SCM roles and processes
• Design Pattern 2: Version Control – automation of base SCM functions (Check-in, Check-out, on-line forms, etc.) to support single state lifecycle CIs (documents, baselines, etc.)
• Design Pattern 3: Multiple States – support multiple teams (Developers, QA, Customers) working on different views (States) of an Application. Also provides for Management control of CIs and their groupings as they progress through multiple States. A robust relational database that can manage these activities
• Design Pattern 4: Build and Deployment – ability to support diverse CIs, command line interface (API), interface to a build management tool, scripting environment to direct the activities of a build and deploy. A robust relational database that can manage these activities
• Design Pattern 5: Parallel Development – advanced user interface including file comparison, version tracking, branching, and merging. A robust relational database that can manage these activities. Still lacking in tools to evaluate the impact of merging.
• Design Pattern 6: Content Management – Graphics based web interface, large grain SCM, and web site deployment tools

SCM is a constantly evolving discipline. This is due to the fact that new software development methods, tools, technologies, and other non-technical forces continually drive the needs of SCM systems to provide functions and features to support them. SCM tools in the future will need to address:

• Geographically diverse development teams will require more remote capabilities
• Outsourcing of different components of an Application to different companies will require remote capabilities as well as security to proprietary CIs within a SCM tool
• More/Better integration with development tools/styles such as Integrated Development Environments, Object Oriented Design, Rapid Application Development, Joint Application Development, Extreme Programming, etc.
• Multi-Lingual support
• Governmental regulation compliance
• Information security
• Process methodologies (CMMi, ISO 9000, ITIL)

But as I stated in the beginning of this whitepaper, planning and process should always be the first and foremost activities of any SCM system.

Acknowledgments

Terry White and Steve Parker for their review and input

References
Configuration Management, the missing link in web engineering by Susan Dart Parallel Configuration Management Process by Meyer Frankel ABCs of a Branching and Merging Strategy by Mario Moreira Jim Johnston has worked in the IT industry for over 20 years. He has held multiple positions including CAD/CAM programming, applications integration coordinator, manager of a application launch center for testing and integration verification of PC applications, and testing architect for major web systems. Jim has been involved with SCM for over 6 years primarily in the areas of enterprise web applications, retail web sites, SCM process development and J2EE build and deployment methodologies.

Read Part 1 at SCM Design Patterns: Paper Forms

Read Part 2 at SCM Design Patterns: Version Control & Multiple States

Read Part 3 at SCM Design Patterns: Build and Deployment

About the Author

Jim Johnston has worked in the IT industry for over 20 years. He has held multiple positions including CAD/CAM programming, applications integration coordinator, manager of a application launch center for testing and integration verification of PC applications, and testing architect for major web systems. Jim has been involved with SCM for over 6 years primarily in the areas of enterprise web applications, retail web sites, SCM process development and J2EE build and deployment methodologies.