The post-mortem most teams do not have
Many engineering teams do one of two things after a serious outage. They skip the review because everyone is tired and the fix is deployed. Or they hold a meeting that drifts into finding out who pushed the bad change, and people leave defensive. Neither produces learning. The same class of failure returns a few months later, usually on a worse day.
A blameless post-mortem is a structured way to turn an incident into durable improvement. It is a core part of our SRE and incident response service, and we facilitate it for critical incidents so your own engineers can concentrate on the substance.
What “blameless” actually means
Blameless is often misunderstood as “nobody is responsible”. It means something narrower and more useful: the review assumes people made reasonable decisions with the information and tools they had, and asks why the system allowed or encouraged the outcome.
Compare two sentences:
- “Dev X deployed a bad config and took down production.”
- “A config change with no validation step could be deployed directly to production, and the failure was only visible to customers before it was visible to us.”
The first ends the conversation. The second gives you three fixable things: validation, deploy safeguards and detection. A review that lands on the first sentence has found a person, not a cause.
Accountability still exists. It lives in the actions: each has an owner who commits to finishing it.
What good looks like
- Evidence-based timeline. Built from alerts, deploy logs, chat and the scribe’s record, not recollection.
- Several contributing factors. Serious incidents rarely have a single root cause. Trigger, latent weakness, missed detection and slow recovery usually all appear.
- Honest on detection and response. How long did it take to notice, to declare, to mitigate?
- Few, owned actions. Three to six meaningful actions beat twenty vague ones.
- Shared. Findings reach the wider team, not only attendees.
The template
A report should fit in a few pages and be readable by an executive in five minutes. Ours follows this structure:
# Incident review: <short title>
Date: <date> Severity: <level> Duration: <start to end> Author: <name>
## Summary
Two or three sentences: what happened and what users saw.
## Impact
Who and what was affected. Customer-facing effect. Budget consumed against the SLO.
## Timeline (UTC)
- 14:02 Error rate begins rising (first alert at 14:07)
- 14:09 Incident declared, commander assigned
- 14:21 Rollback started
- 14:30 Error rate back to normal
## Detection
How did we find out? How could we have known sooner?
## Contributing factors
Trigger, latent conditions, gaps in safeguards, gaps in tooling or knowledge.
## What went well / What went poorly / Where we got lucky
## Actions
| Action | Owner | Due | Ticket |
The timeline values above are an example layout, not data from a real incident. The “where we got lucky” section is valuable: it surfaces near misses that analysis would otherwise miss.
How we facilitate
A facilitator who was not in the incident is useful because they have no stake in the story. Our approach:
- Set the tone. Open by restating that the goal is learning and that nobody will be blamed.
- Walk the timeline. Ask what people knew at each point and why a given choice looked reasonable then.
- Ask “how” and “what”, rarely “who” or “why did you”. The latter invites defence.
- Explore contributing factors. Techniques such as the five whys can help, but use them carefully: a chain of “why” questions can force a false single root cause. We prefer a broader factors map.
- Converge on actions. Each action must be specific, ownable and possible. “Improve monitoring” is not an action. “Add a burn rate alert for checkout latency” is.
- Capture quiet voices. Junior engineers and people not directly involved often spot the weakness everyone else normalised.
Tracking actions so they finish
The main failure of post-mortems is not the meeting but the follow-up. We create a ticket for every action at the end of the session, review the open list on a regular cadence, and report ageing actions to engineering leadership. Where an action is rejected, we record that decision and the accepted risk. Many actions feed other parts of this service: improving runbooks, tightening alert quality or automating a repeated fix as covered in toil reduction.
Decisions and trade-offs
Which incidents to review. Reviewing everything wastes time and breeds fatigue. We set a trigger based on severity and customer impact, and allow a lightweight written note for smaller events.
Facilitator independence. An outside facilitator is helpful when the incident involved senior people or political tension. Once your team has the habit, rotate internal facilitators and use us only for the critical ones.
Public versus internal. Some companies publish customer-facing summaries after major outages. Decide who approves them and what level of detail is wise. This is a business decision, and legal input may be needed for security incidents.
When blameless is not enough. Repeated negligence or deliberate policy violation is a management matter handled separately from the learning review. Keep the two apart.
Common failure modes
- Naming a person as the root cause. Ends the learning.
- Action inflation. A long list nobody finishes.
- No follow-up. The report is filed and forgotten.
- Delayed review. Evidence gone, memories distorted.
- Executives dominating. People stop speaking honestly. The facilitator manages the room.
Delivery and handover
For each critical incident we provide evidence gathering, the facilitated session, the written report and the action tracker. We also leave you the template, a facilitation guide and a short training session so your own leads can run reviews. If you prefer to hand incident operations to a partner, our Managed DevOps engagement includes the review habit.
How to tell it is working
Look at action completion rate and age, repeat incidents of the same class, whether engineers speak openly in reviews, and whether detection and recovery times trend down over successive incidents. A healthy sign is people volunteering their own mistakes because they trust the process. To set up reviews for your team, book a consultation.