Lean Software Development: Leaning Against Perception

[article]
Share URL
 
Summary:
The seven principles of Lean Software Development; eliminating waste, creating knowledge, respecting people, building quality in, deliver fast, optimizing the whole, and to defer commitment are concise principals that appeal to many developers and managers, as they are pivotal in achieving delivery of high quality software systems on time.

The seven principles of Lean Software Development; eliminating waste, creating knowledge, respecting people, building quality in, deliver fast, optimizing the whole, and to defer commitment are concise principals that appeal to many developers and managers, as they are pivotal in achieving delivery of high quality software systems on time.

These principles have a foundation in Lean Manufacturing that had been very successful throughout the 20th Century, which delivered customer value and popularized Toyota's turnaround in the 1980's. So it may not be a surprise that Lean has moved into software development and appears, so far, to be quite successful.

However those new to Lean may perceive some of the practices involved non-intuitive and sometimes contradictory to Lean principles, in particular: 

    • Ceasing Inspections (contradictory to creating knowledge) - for example ‘If we stop code reviews how can we understand how to improve the code and where we are making mistakes?
    • Deferring Commitment (contradictory to Eliminating Waste) - ‘If we leave decisions to the last minute, we will have people sitting around not working!'
    • Knowledge Sharing (non-intuitive to a successful product) - ‘If we share our issues with the customer they will lose confidence in our ability to produce a successful product!' or ‘If we share ideas outside of the organization someone will copy our idea and we will lose the intellectual property and the advantage of time to market!' 

By investigating these three points it can be shown that these practices are indeed beneficial to the organization by driving quality within the development group and increasing customer satisfaction in the software industry.

Cease Inspections
"Cease Inspections" does not mean stop all code reviews. Code reviews are useful and create knowledge. What this concept is telling us is to eliminate large reviews at the end of a development phase.Large reviews invariably lead to bugs being found, but at the expense ofdelaying the project, in some cases up to several weeks or months.

Code reviews need to be performed at a level where waste can be eliminated (another of the seven Lean Software principles). By having frequent reviews, any bugs or improvements discovered can be discussed with the owner of the code. This continual feedback allows the developer to be in the correct mental context to refactor and fix the code faster than if several weeks have passed since he last touched that part of the code base.

By implementing continuous integration with unit tests (and integration tests where possible) and integrating tools such as static code analyzers, dependency analyzers and code coverage software, the mundane parts of Code Inspections that are subject to human error are eliminated leaving the more expensive resource (the Reviewer) to concentrate on the core business domain that these tools are not [yet] intelligent enough to investigate.

By not using the tools available, reviews become de-motivating, overly complex and time consuming (again, waste). These reviews then often simply fade into oblivion, leading to a necessary large review at the end of the project. By only having this large final review many bugs and some feature ambiguity will arise, leading to rework that either extends the time to delivery of the solution or eats into other processes such as User Acceptance Testing. This is precisely what we are trying to avoid. COMMENT1

Another practice that can provide continuous, real-time inspection is pair programming. The code is continually inspected by at least one other person and therefore defect rates are lower and design decisions are questioned at code-time (Build Quality In). Communication is enhanced and more team members share knowledge of the system thus creating knowledge.

Defer Commitment
"Deferring Commitment" means leaving decisions until the last responsible moment. This is non-intuitive to many Project Managers and Developers who normally prepare Project Plans from original requirements. Not to mention all people who are taught by their parents; "Never put off until tomorrow what you can do today!".

We know that developers are asked for estimates regarding work in the future, well these are just that, estimations! At some time early in the process, these estimates become contracts to the customer who are not happy when functionality is not delivered ‘on-time' and developers are working long hours to satisfy the raw estimates that the customer has been promised. These estimates are far more accurate when delivered closer to the time of execution (i.e. implementing a story or feature) and when given in context when the developer knows more about the system they are contributing to.

Functionality also changes over time. Iterative Development is a cornerstone of Agile and one of the concepts that has made Agile practices successful. Prioritizing time-boxed stories so that each iteration (or Sprint) consists of highest priority work that is achievable, has allowed more frequent deliverables to customers. Over the life of the project, requirements change, and can be introduced in between iterations. Allowing customers to defer commitment on features just before the iteration begins, at the last responsible moment, allows them time to confirm that the functionality is still relevant to the solution, or for them to realize that this functionality is now obsolete or less of a priority. This saves the team from performing work that might have otherwise been ultimately rejected.

"Deferring Commitment" does not mean leave everything until the last minute, otherwise the project will be late.

Why hope a design, or even hardware decision, will be suitable weeks or even months before it is absolutely needed? A design decision for a future feature could be totally negated through different decisions in early features when implemented and therefore the time spent in meetings discussing those future features, that are now not required, is waste. Also as a developer, your contextual or domain knowledge will increase as you are working on a project. The more you learn about the domain, the better informed you will be to make a decision

One approach to "Deferring Commitment" is to use Set Based Design. This works as an option when the deadline cannot be moved. One interpretation of Set Based Design is to have parallel development streams so that one team works on the simplest solution with minimum requirements (guaranteed to be delivered on time), while another team works on a system that has more functionality (may be delivered on time, but not guaranteed) and a third team may work on the ideal system (unlikely to be completed by the deadline). The customer will have a system by the deadline and management can wait to the last possible moment to decide which one to implement knowing there is a fall back plan. The idea is to complete the ideal system in the near future, to be delivered to the customer at a later date.

Admittedly, I have rarely seen this implemented. One successful example was at IBM in the early days of E-business (mid to late 1990's) where manpower was not an issue and different techniques were applied to determine the ‘best' way to deliver a project. Most organizations do not have the resources to perform this, do not want the extra overhead (even though it may be incorrectly seen as wasteful), or are fearful the customer will be happy with the current system and not implement the ideal one. Today, this type of work is usually seen early in the project where the customer can decide from multiple prototypes and then drive the way forward through iterative development cycles.

Knowledge Sharing
Sharing knowledge (sometimes referred to as ‘Amplify Learning' in Lean circles) within the organization is commonplace and advocated in every organization.

Personally I believe the best examples of this within Agile methodologies are the Scrum techniques of the Daily Stand Up Meeting and the Sprint Planning meeting. Not only is communication among the team significantly increased, including problems and successes, but if the customer is in attendance they can appreciate more why any delays in functionality (or the project) is occurring. Sometimes we (developers) forget that the customer may have parallel processes running while the system is being created. For example; process change, user training and document creation. Customers understand that Software Development can be extremely complex and would prefer to know of issues sooner as they can then adjust any external schedules appropriately, rather than finding out at the eleventh hour that the project is delayed.

Sometimes perceived Knowledge Sharing leads to waste, another Lean principle we want to avoid. This is primarily the case where organizations are attempting to extend Scrum through the Enterprise performing Scrum of Scrums where some of the projects are unrelated. This policy has forced managers and team leaders to attend irrelevant meetings in the mistaken preconception that the organization is adhering wholly to being ‘Agile'. This has been a factor in recent reports that many Scrum teams are either failing or not getting the benefits they hope to get.

Extending the Lean principle of sharing knowledge outside of the organization,specifically research and especially to competitors, is currently not a common practice. The question is: are there benefits to this?

In the Japanese Manufacturing industry, Ramp;D groups from different organizations have worked together to investigate new technologies, thus sharing costs and eliminating duplicate work. Only when the technology is realized, the same organizations compete in the market place using performance factors and features to differentiate their products.

Executives from a SD organization in the US or Europe may think this as a competitive disadvantage and that innovation should be kept behind closed doors. However we have a great example of this that has existed for several years, Open Source projects. One particular project is Eclipse. Initially conceived at IBM Canada, Eclipse was moved into the Open Source arena where it attracted a large number of contributors. Once released, the IDE gained a large user base and now has commercial competitors such as RAD from IBM, JBuilder from Borland and BEA Workshop, who are all competing on feature sets and price.

This practice promotes optimization in the industry, significantly reducing costs and increasing customer satisfaction. This last point is evident in the current demise of the HD format against Blu-ray for high-resolution video. How many of the 750,000+ purchasers of HD Players wish that Sony and Toshiba (who have now ceased production of HD DVD machines) could have worked together to produce a common standard?

I believe the cultural change is too great for this practice to occur within perceived competing organizations today. A shift is required from competitors fighting for a share in the current market to collaborating to increase the market and compete for a share within this larger market.

Conclusion
It is evident that a convergence of Lean and Agile principles is occurring and the term ‘Lean-Agile' is becoming more widespread (At time of writing Google returned around 32,000 results for ‘Lean-Agile'). The overarching principles of Lean, specifically removing waste and reducing complexity in the organization are really what Agile methodologies are trying to implement at the lower level. Applying Lean thinking to any Software Development practice raises the question of ‘Am I really adding value by doing this?' - whatever ‘this' is.

This article focuses on only three Lean principles; cease inspections, defer commitment and knowledge sharing but by applying these on top of your current practices will give a better holistic view of how your project gains value for the organization and opens the door to the other Lean principle not covered here.

From the number of Lean topics at Software Conferences, articles, and blogs on the subject, the Lean movement is growing. I believe more projects will succeed by utilizing Lean principles. For anyone new to Lean, investigate beneath the surface as even expert groups are still discussing the most efficient ways to implement some of the principles - Lean is lean itself as it keeps evolving to find better practices. To modify a favorite quote "Why is anyone interested in these theories and ideas of management? Because they work!" 

References:

About the Author

Richard Sharpe has been working on software projects for 10 years as a Developer, Consultant and Manager, more recently focusing on improving software quality and process management in the Java industry. Rich has traveled around Europe and the US speaking at various Java User Groups as well as Javapolis and JavaOne on subjects regarding Java Performance issues, Managing Development Teams and Quality and Metrics Initiatives. Various articles have been authored for Websphere Advisor Magazine, JAX Mag (Germany), Agile Magazine and International Developer Magazine on these topics. Currently as a Director for Enerjy Software, he consults with customers on Quality Issues and hosts Enerjy.tv as part of the www.enerjy.com blog, interviewing various industry leaders on various Quality related topics.

About the author

CMCrossroads is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.