Agile software development methodologies grew out of lean principles pioneered in business and industry over the past seventy years, and they are now often referred to as lean’s digital counterpart. By better understanding the philosophy behind lean, developers can gain insight into agile and its uses and pitfalls, making the most of its practice for their team.
Software development is about creating products and IT applications. The traditional process for project delivery is the waterfall model, in which requirements are defined all at once prior to project initiation.
Historically with this model, the typical target project duration is around nine months. In my experience, however, I have never seen a project executed successfully within this timeframe. Invariably, there have to be compromises in the scope, cost, or quality of the project in order to ensure the delivery schedule doesn't change. The most common reasons I have seen for these compromises are poor requirements understanding, incorrect estimations, tangled team dependencies, or a change in business process by the time the project is delivered.
Agile development has evolved as an effective methodology by applying lean principles to the standard (and often inefficient) waterfall development process. Thus, agile methodology applies lean business practices to software development in order to improve productivity, quality, and the ability to implement configuration management applications such as DevOps.
Lean software development can be summarized by seven principles that are very close in concept to lean manufacturing principles:
- Eliminate waste: The product manager works with the team in breaking up stories and determining business value. He also works with the engineering team to get the time cost for completing each story. Story priority is decided mostly on these two values. The Scrum team takes high-priority stories, which effectively eliminates waste.
- Amplify learning: The agile retrospective meeting is a valuable resource to carry over learning to the next sprint plan.
- Decide as late as possible: Sprints of two to four weeks will give the necessary flexibility to make decisions later.
- Deliver as fast as possible: Once the story is moved from the product backlog to the sprint backlog, the turnaround time for delivery is short.
- Empower the team: Because the Scrum team is selecting which stories will be pulled from the product backlog to the sprint backlog using story priorities, management is not making decisions on what should be part of the sprint backlog. This empowers the actual team to make decisions without those further from the work interfering.
- Build integrity: Agile core values ensure team member integrity by making them truly necessary parts of the team, resulting in buy-in and pride.
- Work holistically: The release plan makes the Scrum team approach projects holistically because it provides a bird's-eye view of the value of each activity while still maintaining focus on the sprint.
The success of an agile implementation depends a lot on organization mindset, especially leadership maturity. Using enabling tools, such as JIRA Agile with Confluence, is also recommended in order to record all necessary artifacts. Continuous integration, continuous delivery, and test automation complement the value agile creates. Effectively, a Scrum framework reduces the failures highlighted above as much as possible.
However, there are still problems with an agile Scrum framework.
For one, it is focused on the engineering team. Though the product owner is from the product marketing team, there is no established framework for how the product backlog should be groomed. Usually there are multiple product owners with requests that need to be included in the product backlog. To address product log grooming, I recommend a kanban process. This way the input from service engineering into the product backlog remains an open area.
Secondly, it also can be difficult to scale this framework to a matrix or larger engineering project teams. Agile recommends six to eight people to a team. However, at most global and matrix organizations where I have worked, projects usually are built by twenty- to fifty-member teams. Some principles of Scrum work to some extent within engineering, but when representing an engineering team in cross-functional project meetings (with marketing, engineering, systems, service engineering and operations, sourcing, QA, risk analysis, and program or project manager), agile and Scrum processes aren't understood. In these situations adaptability is a key factor. You have to prepare yourself differently for different meetings.
For example, sometimes an OS announces end-of-line or other strategic decisions that drive changes and product platform migrations. Remember that to the customer, it doesn’t make any difference; they only care about application functionality. As per agile, teams need to focus on changes that make a difference to the customer. Nevertheless, these issues should not be neglected.
There can be side effects to doing design in the sprint, especially for the products that need to be scalable in nature. Platform design is critical in the long run, and in the rush of delivering shorter cycles, this should not be compromised as it can prove very costly in the long term.
There also can be issues with Scrum when it comes to products having incremental releases to multiple instances. Continuous delivery works well for a single instance, as in Facebook’s push model. Releasing to external teams after every sprint makes sense, but imagine updating a high-volume install base. Some argue that companies like Microsoft address this by releasing patches or service packs very frequently. However, this practice isn’t covered under the standard agile framework; instead, updates are part of the continuous delivery scope. Additionally, upgrading systems that are under regulatory surveillance is a different ball game.
In short, though no framework is a panacea, in a constantly changing digital landscape, the adaptability of agile is a major strength. While CEOs, CTOs, and other management can adapt more long-term and strategic work processes, agile as a digital form of lean business is certainly here to stay. As lean emphasizes making improvements to existing processes to satisfy different challenges and refine frameworks, it will keep evolving. The best way to take advantage of the new directions this lifecycle will take is to stay centered on the original lean philosophy out of which it arose.
