Development managers at large organizations with monolithic application lifecycle management (ALM) stacks once had it good. Change and configuration management tools were well-integrated, played nicely with one another, and when they didn’t, there was someone to call. But over the past decade open source tools have moved in, popularized by developers’ demand for lighter-weight collaboration and change management facilities. At a cheap or free price point, these tools were often deployed without the IT department’s knowledge or approval. This article examines the benefits that open source ALM and tools has brought to developer productivity, and discusses the problems that many organizations are experiencing with the lack of integration between these tools and the rest of their ALM and CM stack.
Thanks to a combination of resource constraints, a preference for using open source technologies for open source development, and the common desire for developers to sharpen and extend their own toolset, the past decade has delivered major innovations on the front of open source ALM tools. As open source project portals have matured, so have the ALM tools that support them. Consider the scale of the open source ALM stack that supports Eclipse.org: 33M lines of code in the 2010 last coordinated release (733 installable features/components), 330K total Bugzilla reports (3K new each month), 107K Bugzilla users (5K active last 30 days), 1K committers (400 active), 300 projects and 173 Hudson build jobs. Add to that dozens of interdependencies between Eclipse projects and other open source projects such as popular Apache libraries, and downstream dependencies from commercial offerings built on Eclipse. ALM on Eclipse is managed entirely with open source tools including Bugzilla, CVS, Git, Hudson, MediaWiki and Mylyn.
The 1,948 respondents to the 2010 Eclipse Community Survey provide an overview of the degree to which open source ALM tools have percolated into the commercial software development process. The striking pattern is that the core open source ALM tools, when combined, have the market lead in each of three key ALM categories visible in the figure below. While surveys of this sort are always skewed towards the type of developer who bothers to answer surveys, this result remains indicative of a shift in application development practices and open source ALM market share. Notably only a small fraction of the survey respondents were directly involved with Eclipse, and half were from organizations with more than 100 employees. In 2011, this open source trend will continue as these tools percolate into the ALM stacks of more conservative organizations. A degree or two of separation from their open source enthusiast counterparts, many of the developers at those organizations will not be familiar with the Git tool before a DVCS solution is force fed to them.
The attraction to open source ALM is not just price point, but the amount of innovation that has been created by open source developers building tools to support their own productivity patterns. The ecosystem of extensions that forms around popular open source projects has been another key driver of adoption. In addition to these developer friendliness and the availability of extensions, organizations with long application lifecycles are attracted to open source ALM tools because a suitable open source license gives them confidence that they will be able to access and extend the knowledge base represented by their issue tracker ten years from now, when the landscape of ALM vendors will have changed.
Open source ALM tools are built on developer-centric principles. Transparency is favoured over hierarchy, with very task and bug on Eclipse and Mozilla being editable by anyone. Asynchronous collaboration and a constant use of the task or issue tracker ensures a the capture all discussion relevant to changes in the software. Modularity and framework boundaries are naturally aligned with team boundaries, allowing API layers to facilitate dependencies and collaboration. But major gaps exist in the today’s open source ALM stacks. Responsiveness to the community often takes precedence over planning, and as a result there has been a distinct gap in project management features within the open source ALM tool space. There is also no single integrated open source ALM stack, instead open source projects glue together their own best of breed solutions, and layer customizations on top, as is the case with the tools built on the Eclipse Bugzilla repository needed to support the Eclipse Development Process. Integration with requirement, product, project and portfolio management tools is generally non-existent, as the planning cycle of open source projects rarely involves that level of sophistication. Quality management tools are also a major gap.
While open source developers will continue working very happily with their limited open source ALM tool set, this impedance mismatch with the ALM needs of large-scale organizations spells major problems for those expecting a straightforward integration of open source tools with the rest of their ALM stack.. The mismatch exists between both the toolset and the cultural differences between open source and enterprise development. There is enough of a gap in the toolset that organizations already deploying open source tools ALM at the enterprise scale have needed to set up their own ALM tool engineering teams. These teams create point solutions for authentication and access control, provide third-party ALM tool integrations, and implement support for features such as linking existing Project Portfolio Management (PPM) tools. Due to the pace of change in open source ALM tools, they are fighting a losing battle. Large organizations that fail to integrate already deployed open source tools into their existing ALM and PPM infrastructure will see a dramatic reduction in the predictability of their development process, since their process relies on a connectivity between development and planning tools that was present in the more traditional ALM tool stack.
There is hope. The benefits of open source ALM tools are fundamental. The ease with which they allow developers to work makes engineering happier and more productive. The velocity of successful open source projects demonstrates how effective these tools are at supporting the delivery of high-value software that is responsive to the needs of the customer community. On the flipside, enterprise ALM tools provide management and planning facilities critical for the predictability of delivery as well as the planning features necessary for software delivery at scale. These two worlds must be integrated into a cohesive whole, especially as more Agile teams find themselves at the intersection of open source and enterprise ALM. Failing to automate the connectivity between the new breed of open source ALM tools and the established enterprise ALM tools will mean that organizations falls back to less reliable communication formats such as excessively long email threads, more tedious meetings, and the excessive wearing office carpets as stakeholders walk back and forth between cubicles. Whether driven by the deployment of Agile or the acceptance of open source tools in the ALM and CM stack, 2011 is the time to reconnect the software planning process to a new breed of tools that support lean and high velocity delivery, to connect them to the planning and management tools needed for predictability at scale, and to start bringing the benefits of open source development practices to the enterprise.
