Managing Quality Challenges in Distributed Agile

Small to mid-sized organizations developing software products often struggle to grab their share in the marketplace by juggling with release dates, product quality, innovation, and pricing. Sometimes, product quality is kept on the back burner due to the fact that these organizations also need to sustain the interest of their investors by demonstrating product's profitability. An ideal product strategy demands a perfect balance between innovation, time-to-market, cost-effectiveness, and overall quality. This cannot be achieved without having an effective product lifecycle that is capable to cope with continuously changing requirements since quality primarily deals with realization of business requirements.

Trends in product development are rapidly changing over the past decade. In today's prevailing trend, product teams no longer sit together in the same location to develop software products. Product firms have begun to diversify in different geographies for several business reasons at strategic level. Thus, functional groups within an organization face a dire need for robust processes, framework, and platform that can optimize the distributed product development.

Need for Agile Techniques in Product Development
In product ecosystem, there is an ever-changing need to optimize the way product firms deliver their products with desired emphasis on innovation, quality, and cost. The ability to make continuous significant changes is one of the biggest differentiators of software product development from other engineering disciplines. In order to fulfill this need, product firms continuously seek optimum methodologies such as Agile Methodology. Agile Development Methodologies promote responding to change, by balancing order and chaos.

Agile software development approaches promote high levels of visibility, predictability, and quality by adapting an iterative development approach that promote:

• Evolutionary development of software
• Focus on values and principles, most important of these being - embracing change, delivering business value early and often, and remaining people-focused
• Real-time face-to-face communication
• Working software as a primary measure of success

Agile Development for Distributed Teams
Agile software development is referred to as Distributed Agile, when development teams are completely or partially spread across different geographical locations. This needs molding some of the traditional agile practices which typically assume collocation, emphasizing face-to-face communication and real-time collaboration. This can be done by building a:

• Robust team communication infrastructure
• Knowledge sharing and collaboration setup
• Scalable engineering platform
• Information exchange mechanism

Various agile principles such as Collective Code Ownership, Continuous Integration, Test-driven Development, and Pair Programming demand scalable engineering platform, which should be accessible to all the distributed teams. For instance, in order to exercise collective code ownership, all the developers spread around different locations, should be able to access the entire system. This can be accomplished when a centralized source code repository is accessible across different locations without performance bottlenecks. The success of the distributed agile development is substantially dependent on the way teams collaborate on real-time basis. In this situation, teams should avoid depending heavily on the use of E-mails for knowledge sharing. This is one of the major reasons as to why wiki and Web-based collaboration systems are gaining popularity. The use of IM tools, NetMeeting, and video conferencing can certainly enable effective communication between the teams.

Analyzing Quality Related Challenges in Distributed Environment
When product teams spread across different locations work together, they inevitably encounter various challenges in day-to-day operations. In order to realize these challenges, we performed a study on six different product development teams within the organization catering to our partners in the off-shore product development model. Our goal was to identify key issues in the distributed environment and establish best practices.

All the product development teams were spread across maximum three locations. In almost all cases, Product Management was performed in United States while Product Engineering was spread across multiple locations in US and India. The size of these teams varied from twenty to sixty members. In order to understand practical challenges, we prepared an audit questionnaire consisting of typical questions addressing the key process areas of product lifecycle. For example, maturity of process adherence, adoption of agile principles, and implementation of engineering platform to address the needs of distributed teams.

In order to facilitate this study further, we traveled to these locations and investigated routine problems at all levels. Over the period of three months, we informally interacted with customers, product managers, development managers, engineers, business analysts, and architects. The main results of our analysis are summarized below:

• Rapidly Changing Requirements: Business Analyst, On-site, and Off-shore engineering teams were not on the same page in terms of scope clarity and updated requirement. In some cases, the off-shore team did not properly capture the most recent changes.
• Unrealistic Estimates: Effort variances between estimated and actual were as high as 80%.
• Uncertain Progress Tracking: All stakeholders were not aware with the status of completion of on-going tasks in different locations. Due to this reason, managers were struggling to determine actual progress.
• Showstoppers Found After Release: A few critical scenarios from the high priority business requirements missed during the final testing.
• Missed Defect: In some releases, approximately 10% defects was found in later iterations, while these defect existed from the beginning.
• Communication Bottleneck: Due to hierarchical communication, a few members in some of thee team were becoming bottlenecks.
• Communication Overhead: Repetitive conference calls between teams across locations were affecting productivity.
• Lack of Subject Knowledge: Expected behavior of functionality was known to a few team members; sometimes, information stored in their mailboxes was not shared. Due to this, subject matter experts were repetitively involved in communicating about the same topic. Also, subjectivity of test cases required frequent clarifications from various team members.
• Code Merge Problems: Due to the use of different source control tools, i.e., MS VisualSource Safe and CVS in different locations, code integration was not reliable.
• Broken Fixes: Due to inconsistent check-in practices, simultaneous defect fixes done by the developers from different location were not properly updated in source code repository.

The second phase of the study involved performing gap analysis on each process area. We categorized all findings based on essential parameters such as collaboration, visibility, processes, and tools. After performing root cause analysis on main results, we concluded on the four key solutions to face these challenges:

• Dealing with Dynamic Requirements:We found that lack of scope clarity, subject knowledge, and communication overhead primarily arose due to the fact that distributed teams were not equipped to keep pace with rapidly changing requirements.
• Creating Requirement Tracker: The quality-related problems such as 'showstoppers found after release' and 'missed defects during testing' were occurring due to team's inability to ensure test effectiveness with continuously changing requirements. To meet this objective, an effective traceability between business requirements and other artifacts was required.
• Implementing Continuous Integration: Ability to effectively manage the code base of a product is challenging when developers are sitting in multiple locations. We found that successful implementation of code integration was the solution to various code integration problems.
• Enhancing information exchange with Product Dashboard: We found that managers, leads, and engineers needed better visibility into the product lifecycle without extra efforts. The major cause of problems related to communication, estimates, uncertain progress tracking, etc. was unavailability of a product dashboard.

In the third phase of this initiative, we established best practices to overcome the above-mentioned challenges. Since methods and tools work hand in hand, we implemented scalable platform to support the best practices.

Build a Mechanism for Dealing with Dynamic Requirements
Appropriate realization of requirements seems to be one of the biggest challenges of Distributed Agile development. This is also one of the biggest causes for software failing in distributed product development. Thus, product engineering teams need to adopt best practices that are capable of accommodating dynamic requirements coming from different stakeholders at any given point in time. A business requirement may be coming from a customer, product manager, developer, tester, or support team. Further, there may be different modes of communicating requirements, e.g., via instant messaging, stand-ups, market requirement document, verbal communication, e-mails, etc. One of the important roles of quality engineers at this stage is to validate business requirements and ensure that all the market requirements are well satisfied. It becomes necessary that all stakeholders collaborate to keep the requirements-related defects away. Another important aspect of requirement testing is to be able to determine missing pieces of requirements. Agile practices recommend seeking early feedback by involving customers from the beginning since they are the best judge of their business needs. Moreover, early involvement of customer also enables product teams to keep pace with continuously changing needs and wants of their customers.

All these objectives can be successfully attained by designing a requirement management system that allows dynamically storing and tracking business requirements. To help facilitate distributed environment, this system was accessible to various teams over the Web so that all stakeholders can collaborate on real-time basis. Such a dashboard promotes collaboration amongst various stakeholders and helps to track the changes and the progress. It allows a common view for all stakeholders. It also helps in building a knowledge-base of requirements over a period of time. This knowledge-base provides distributed teams an opportunity to leverage product knowledge without depending on subject matter experts.

Creating an Integrated Tracker
Ability to properly derive test cases from raw requirements is important in product testing. In traditional approaches testing is typically done with the purpose to find defects. We believe that in Agile approaches, software testing in a simplistic sense is about realization of requirements. All the tasks associated with fulfilling the delivery of the requirement needs to be finally mapped, inter-linked, and driven by its priority. These tasks can be associated with activities like writing use cases, code reviews, writing test cases, test automation, etc. This also helps in proper collaboration in scenarios of rapidly evolving requirements.

In addition to test cases and use cases, requirement mapping is also necessary for test results, defects, and code changes. When a defect is fixed, one should be able to instantaneously map this fix with associated business requirement. This mechanism also helps improving effectiveness of testing since one can easily figure-out the more stable or unstable business requirements. This information allows product teams to pay more attention to business requirements that are mission-critical and most susceptible.

Establishing traceability for business requirements, use cases, test cases, test results, and defects requires robust method and tool. This can be achieved by designing a common framework that enables integration of various repositories (e.g., business requirements, test cases, test data, automation scripts, and test results) with issue tracking system. The workflow of this framework should be designed so that it provides tracking from one artifact to other on a single click.

In order to ensure effectiveness of tracker, one should adopt suitable nomenclature and configuration for various iterations and associated artifacts. The requirement tracker should also be simple so that traceability and accountability are easy to maintain and the visibility requires no extra work for the leads and managers.

Implement Continuous Integration
Continuous Integration (CI) mechanism facilitates integrating changes to the code base continuously by frequently running distributed builds and tests, checking and improving code quality, all the while making progress towards successful product development.

Successful implementation of Continuous Integration not only prevents code merge from the last minute surprises, but also progressively ensures code stability to a great extent. Effective implementation of Continuous Integration requires a robust framework to automate the release management process. This framework should allow enabling the integration of various tools such as Build Automation, Repository Browsing, Version Control, Issue Tracking, and Reports. For distributed teams, it is essential that the source code repository is centralized and unified tools are used.

In the Continuous Integration Framework, build generation can be automated using a suitable tool such as Quickbuild. The automated scripts of the Build Automation Tool should periodically build, deploy, promote, and test the update code when checked into the code repository. Any failure should be immediately captured, and subsequently an associated issue should be automatically created in the issue tracking system. For enhanced visibility into the progress of build process, build scripts should be integrated with product dashboard to continuously display the build reports. The integration of the Build Automation Tool with the Version Control, Repository Browsing, and the Issue Tracking tool leverages transparency into the development process. For instance, the Issue Tracker displays the list of modified files for a defect, when the fixed code is checked into the code base.

Product Dashboard as Information Radiator
In addition to requirement traceability, Distributed Agile development needs a common view, which allows assembling product engineering processes together. This requires a Product Dashboard that provides a holistic view of the entire product lifecycle by reconciling information from requirement management, test case management, issue tracking system, etc.

The concept of Product Dashboard helps tremendously in distributed agile product development since it provides great transparency from all perspectives of product management. Product Dashboard should be designed so that it gives real time visibility into the progress of product lifecycle at any given point in time. For instance, product roadmap should comprise the list of new features and targeted issues for a particular release. Dashboard view should be configurable so that it caters to different roles within product development. For instance, a development manager might be most interested to view product roadmap, release calendar, actual progress, estimates, work load, burn down, and burn up charts, etc. On the other hand, quality engineers need to view information such as assigned tasks, current/ forthcoming releases, test suits associated with planned features for a release, test coverage reports, to name a few. With Product Dashboard, product teams can save a lot of extra efforts needed to manually consolidate this information from various sources. Hence, an effectively designed role-based dashboard boosts productivity of the distributed product teams at large.