TimeLine Postmortems

You've got to accentuate the positive
Eliminate the negative
Latch on to the affirmative
Don't mess with Mister In-Between
-- lyrics by Johnny Mercer

Inevitably, something goes wrong on projects. Ideally, something else goes right. On the next project, you want to prevent the failures from reoccurring and reproduce the successes. This is why you need postmortems.

Most teams don't use postmortems. The teams that do don't always get value from them. Frequently, postmortems have a habit of either turning into "the blame game" or whitewashing mistakes. A bad postmortem can create dissention and institutionalize mistakes.

A good postmortem builds team rapport, rewards project successes, and improves the next project after it. Good postmortems are rare, but they don't have to be. Whether you're introducing postmortems to a team that doesn't use them, rescuing a failed postmortem process, or improving a successful one, timeline postmortems can help.

The complete timeline postmortem process takes three to four hours of team time plus a little preparation for the moderator. The process produces results that are immediately applicable and focuses on primary benefits. You'll be able to introduce incremental change that the team can adapt to rather than reinvent the entire development process.

In keeping with this philosophy, the process elements can be separated and introduced one at a time.

Process Outline
The basic steps of a timeline postmortem are:

1. Create the project timeline.
2. Walk through the timeline, identifying successes, failures, and "themes" of the project.
3. Group successes, failures, and themes by project phase.
4. Adjust your software process to address the failures and recreate the successes.
5. Only implement process changes that affect the immediate future.
6. Revisit the postmortem when subsequent projects enter a new phase.

Key Elements of the Process
Here are some valuable things to keep in mind:

• Separate actions from interpretation and stay away from fuzzy "why" questions like "Why did that fail?" Instead, ask "what" or "how" questions like, "What happened to cause it to fail?" or, How did it fail?"
  
  Creating a timeline of project events allows the team to talk about events and process without pointing fingers. Getting agreement on the timeline first, without any analysis, keeps the later discussions from devolving into "did not/did too" exchanges.
  
  In addition, the timeline becomes the "official truth" for the team. Shared history increases "teamly-ness." Just the act of creating the timeline can resolve esprit issues after a difficult project.
   
• Give equal attention to each part of the project. Postmortems can easily shift focus to the pains of the final phases of a project because that's the freshest in the team's minds. A good timeline will remind the team of successes and failures throughout the entire project.
   
• Do a postmortem right after the project is over. You can use the timeline process before that if a project is stalled or failing. Waiting until another project begins provides a useful perspective.
  
  On failing projects, set up a recommitment meeting when creating a timeline. Doing so implies that any issues are part of the past and focus should remain on the results rather than the difficulties.
   
• Always have an impartial moderator/facilitator. This moderator only asks questions and writes down answers the team agrees on. The moderator should not be contributing opinions.
  
  If your company doesn't have someone impartial, hire an outsider.
  
  If you have to lead the postmortem yourself, create a clear distinction between providing facilitation and providing your opinion. One good approach is to stand and sit when in a specific role: standing when asking for information and repeating what the team says, sit when giving your opinions. Have someone ready to interrupt if you step out of role.
   
• Describe events and proposed process in terms of behavior, not intent. What is the trigger? Who does what? What causes it to end?
   
• Address technical decision-making as well as traditional development process. Be sure to discuss technical decisions in terms of criteria and results. Watch for finger pointing and dogmatism in this area.

Preparation
To implement the process, you'll need a few items:

• One Impartial Moderator
  Too many postmortems are run by project managers or executive sponsors. In this postmortem environment, the team members are unlikely to be honest about successes and failure. When the facilitator is not a manager, he may have a preferred software process methodology to push, which will skew the postmortem results.
  
  If possible, get someone outside of the group--or even outside of the company--to lead the postmortem. The facilitator needs a basic understanding of the development process, but doesn't have to be highly technical or intimately familiar with the project. In fact, a degree of unfamiliarity will help.
   
• A (Mostly) Blank Project Timeline Template
  Begin with some basic research: When did the project begin? When did the team cross major milestones, like first QA drop and final ship? Were there any personnel changes during the project? Major holidays?
  
  Put all of these on the project timeline to begin.
   
• Project Phases
  Use project phases to contextualize successes and failures. During implementation, we focus on the current and upcoming phases of subsequent projects.
  
  Phase definitions will depend on your team's development methodology. A typical waterfall project may have specification, design, development, and testing phases. An iterative project may have several iterations, with specifications, design, and test in each one. Be descriptive -- not prescriptive. Phases may overlap or repeat.
  
  During the meetings, the team may adjust the definition and dates for the phases. This is normal. Put a pass at the phases on the project timeline.
   
• The Whole Team
  Gather as much of the team as possible for the first two meetings. Include at least development, QA, and product management, as well as any support, release, documentation, or other groups as may be appropriate. Managers and executives are useful in the first meetings, but you should be prepared to send them out of the room if people seem reluctant to talk around them.

Timeline Creation
Sixty to seventy-five minutes, whole team.

This meeting establishes the project timeline.

1. Put the timeline on a whiteboard for all to see. Leave plenty of room for adding events and activities. 
2. Establish major dates for key events: specifications begun, estimates delivered, first code fork, first delivery to QA, project completion (if possible), etc. 
3. Get buy-in on the project phases. 
4. Add project activities and events. Use language like, "During the design phase, what else happened?" Mark the features affected and when. 
5. For each event or activity, ask, "Does everyone agree that this happened?" Watch the team when you ask this question; look for nods or head shakes. Agreement is more important than precision.

Keep this meeting to an hour. An hour should be sufficient to get a level of detail appropriate to the length of the project.

Do not include any analysis in this meeting. No one should say "why" anything happened, or state any consequences of anything.

Project Analysis
Sixty to ninety minutes, whole team.

You can run the analysis meeting right after the timeline meeting, but put at least a ten-minute break in between so everyone can relax for a moment. Get yourself a drink of water.

When the meeting reconvenes, lead the team through the project timeline from start to finish. Identify key themes, successes, and failures on this project.

Successes? Failures? Themes? Let's define some terms:

Success is something that reduces risk or improves the project's result.

Failures increase risk or reduce the project's success. Definitely include failures that are repeated from project to project. Those are the best to address. Some teams don't like to use the word "failure." This isn't the time to use words like "challenge" or "opportunity." If you need to use a different word, make up something, like "lamppost."

Themes are neither successes nor failures. They provide context for successes and failures. Often, a success in one project would be a failure in another project with different themes. If half of the team thinks something was a success and the other half thinks it was a failure, then it's probably a theme.

Key points to the analysis meeting:

• Work from the timeline.
• Ask questions by phase, but be prepared for people to offer comments on other phases.
• Keep separate lists for successes, failures, and themes. I use three flip charts taped next to the whiteboard. (If you use flip charts and whiteboards together, use whiteboard markers for both.)
• Use behavioral questions only. "In this phase, what worked or didn't work? What was different from previous projects?" Do not ask "why" questions.
• Successes, failures, and themes should all be described in terms of "When X, then Y, which caused Z." Ask for clarification on any comments you don't understand. If a team member says, "We didn't follow the bug process," ask when and whether that was a success or a failure.
• Repeat every success, failure, and theme before writing it down. Get agreement on the wording from the team before listing the item.
• Write down the exact words you repeated to get agreement. Do not translate.
• Keep the meeting moving; try limiting it to an hour.
• Watch those who don't participate. Ask them directly if they have any comments. Language like, "What worked or didn't work for you?" is often effective. Be willing to wait in relaxed silence for an answer.

After the analysis meeting, deliver a document containing the timeline and the project analysis. Organize successes, failures, and themes into project phases. Give the team a few days to digest it.

Process Design
Sixty to ninety minutes, interested team members.

Get the leads and managers as well as interested individuals. This is not the meeting to force people to attend who don't want to participate. Try to get someone from every discipline.

Group successes and failures into any obvious categories: intra-team communication, scheduling, making technical decisions, etc. For each category or item, identify the project phase to address it. Then identify existing processes that affect it. Subsequently, for each phase, develop or adjust processes to address the highest-value items.

When designing a development process, document all of the following requirements:

1. A cue to start
2. An operation or behavior
3. A way to test progress
4. An end point
5. Process(es) to switch to

Here are some tips about process design:

• You may not be able to address every success or failure. Focus on phases and changes that will be immediately useful on other projects.
• Process will change. No team follows every process it expects to and no team follows every process the same way every time. Since you know that, don't design in too much detail.
• If the team is split on a process, just pick one side. Let the team try it one way and change if needed. Be willing to be wrong.

Implementation of Process Changes
When the next project starts--if it hasn't already--you're ready to implement new process. Only implement pieces that affect the project phase you're on. For example, don't spend time re-tooling your release process before the specifications are written. When each project phase ends, review the previous project's postmortem and implement process for the new phase.

Write down new processes and post them in a public place. Always refer team members to a written description of the process.

If something doesn't work, acknowledge it and move on. Make a change if you have to. If you can muddle through with a broken process, it's best to save the redesign for the next postmortem.

Always remember that the process is not the product. You have software to ship.

No matter how successful your current project is, I hope your next one is even better. And may you never mess with Mister In-Between.