How to Write a Blameless Postmortem: Template and Examples (2026)

Oct 20, 2026 · 15 min · Ajay Kumar

This guide is for the engineer who has just been told "you own the postmortem" for an incident that ended an hour ago, and for the lead who wants every team writing them the same way. By the end you will have a repeatable process: when a postmortem is required, how to build the timeline from evidence rather than memory, how to write contributing factors instead of hunting for one root cause, how to turn findings into action items that actually close, how to run a 30-minute review, and a copy-paste markdown template you can drop into a repo. The commands were checked against git 2.54, GitHub CLI 2.102 and kubectl 1.37; you need nothing more than a Git repository and read access to your deploy history, logs and paging tool. If you want the wider context on tools and on-call in 2026, I wrote that up separately in incident management in 2026.

Step 1: Decide when a postmortem is required

Write the triggers down before the next incident, not during it, otherwise every severity call turns into a negotiation about paperwork. Google's SRE book chapter on postmortem culture lists sensible defaults: user-visible downtime or degradation beyond a threshold, any data loss, any on-call intervention such as a rollback or traffic reroute, resolution time beyond a threshold, a monitoring failure (a human found it before the alerts did), and any stakeholder asking for one. Atlassian's incident handbook is stricter and simpler: severity 1 and 2 incidents always get one, everything else is optional. PagerDuty's public response docs say the same for SEV-1 and SEV-2.

Put your version in the repo next to the template so nobody has to remember it:

# postmortems/policy.yaml: read by humans, and by the bot that opens the doc
required_when:
  - severity: [SEV1, SEV2]
  - data_loss: true
  - customer_visible_minutes: ">= 15"
  - detected_by: human          # monitoring missed it
  - rollback_or_failover: true
optional_when:
  - severity: [SEV3]
  - near_miss: true             # nothing broke, but only by luck
review_meeting:
  SEV1: "within 3 calendar days"
  SEV2: "within 5 business days"
priority_action_slo_weeks: 4    # Atlassian uses 4 or 8 depending on the service

Near misses are worth including. They are the cheapest incidents you will ever learn from.

Step 2: Assign one owner and set the clock

One person owns the document; many people contribute. Google's SRE workbook lists "several owners" as an anti-pattern because it means nobody drives it. In PagerDuty's process the incident commander names the owner at the end of the call, and the review meeting is booked within three calendar days for a SEV-1 and five business days for a SEV-2, before the writing starts. Booking first creates the deadline.

The owner's job is to gather evidence, draft, chase reviewers and file tickets. It is not to fix everything; PagerDuty is explicit that the owner creates follow-up tickets but is not responsible for resolving them. Atlassian adds named approvers per service, usually the managers who own the affected systems, whose approval means they agree with the findings and accept the proposed priority actions.

Scaffold the file so every postmortem starts from the same structure. This script creates a dated file from the template in Step 9:

#!/usr/bin/env bash
# new-postmortem.sh: ./new-postmortem.sh INC-2026-0142 "Checkout API 5xx after config push"
set -euo pipefail
id="$1"; title="$2"
day="$(date -u +%Y-%m-%d)"
slug="$(printf '%s' "$title" | tr '[:upper:]' '[:lower:]' | tr -cs 'a-z0-9' '-' | sed 's/^-//; s/-$//')"
out="postmortems/${day}-${slug}.md"
[ -e "$out" ] && { echo "exists: $out" >&2; exit 1; }
sed -e "s|{{ID}}|${id}|g" -e "s|{{TITLE}}|${title}|g" -e "s|{{DATE}}|${day}|g" \
  postmortems/TEMPLATE.md > "$out"
echo "created $out"

Running it in my scratch repo:

$ ./new-postmortem.sh INC-2026-0142 "Checkout API 5xx after config push"
created postmortems/2026-10-20-checkout-api-5xx-after-config-push.md

Step 3: Build the timeline from evidence, not memory

The timeline is the spine of the document, and memory is the worst source for it. People compress, reorder and rationalise within hours. Pull timestamps from systems first, then interview people to explain the gaps. Use UTC everywhere; mixed time zones are the most common timeline bug I see.

# Incident window in UTC
FROM="2026-10-14T09:00:00Z"; TO="2026-10-14T13:00:00Z"

# What merged to main in the window
git log origin/main --since="$FROM" --until="$TO" \
  --date=iso-strict --pretty=format:'%ad %h %an %s'

# What CI/CD ran (GitHub CLI 2.102)
gh run list --created "2026-10-14" --limit 200 \
  --json createdAt,displayTitle,conclusion,headSha,event

# Kubernetes warnings (events expire after 1h by default, so capture them DURING the incident)
kubectl events -A --types=Warning > "incident-events-$(date -u +%H%M).txt"

# Host logs for a unit, using a UTC-suffixed time range
journalctl -u nginx --since "2026-10-14 09:00:00 UTC" --until "2026-10-14 13:00:00 UTC"

The kube-apiserver keeps events for one hour unless --event-ttl is raised, which is why the incident channel should have a "capture events" step in the runbook. Then export the incident channel and the paging tool's log entries (acknowledge, escalate, resolve) and merge everything into one table: time, source, what happened, and a link. Cloudflare's 18 November 2025 postmortem is a good model: a table of UTC times with a status and description per row, running from the 11:05 access change to full recovery at 17:06.

Write timeline entries as facts with sources, not interpretations. "11:32 Alice restarted the bot service (Slack link)" is useful. "11:32 Alice mistakenly restarted the wrong service" is a judgement, and it belongs nowhere in the document.

Step 4: Write contributing factors, not a single root cause

The trigger is what set the incident in motion. The contributing factors are the conditions that let a routine trigger become an outage, slowed detection, or slowed recovery. Google's workbook example separates the trigger (a manual re-run of a workflow) from the underlying flaws (a non-idempotent operation with no safety check) and lists other issues that worsened impact under "things that went poorly".

Two public postmortems show why one root cause is not enough:

  • Cloudflare, 18 November 2025. A ClickHouse permissions change made a query return duplicate rows, which roughly doubled a Bot Management feature file. The proxy had a hard limit of 200 features (about 60 were in use), so the oversized file caused a panic and 5xx errors across the network. The permissions change was the trigger. The contributing factors include treating internally generated config as trusted, a hard limit that failed closed, and fast global propagation. Each one gets its own fix.
  • GitLab, 31 January 2017. In the GitLab postmortem, an engineer removed roughly 300 GB from the primary database instead of the secondary while rebuilding replication. The deletion was the trigger. The outage lasted about 18 hours and lost about six hours of data because none of four recovery methods worked: pg_dump backups had been silently failing (9.2 tools against a 9.6 database, with failure emails rejected for lack of DMARC), Azure disk snapshots were not enabled for the database servers, and nobody owned testing restores.

Group factors under neutral headings so the analysis covers more than the code: detection, change process, tooling and limits, recovery, people and load (who was on call, how tired, how many concurrent incidents), and communication.

Step 5: Use the 5 Whys carefully, or ask "how" instead

The 5 Whys is still the default in many templates, including Atlassian's, and GitLab used it in 2017. It works for a linear mechanical fault. It struggles with real incidents for the reasons John Allspaw set out in The Infinite Hows: "why" questions drift towards who, they force a single chain through what was really a web of conditions, and where you stop asking is a choice that decides what you "find".

You can see the limit in GitLab's own analysis. It needed two separate chains, one for why the site went down and one for why recovery took so long, and the second chain is the one that produced most of the valuable fixes. A single 5 Whys would have stopped at the deletion.

Better interview questions are about how the situation looked from inside it:

  • What were you seeing at that moment, and where?
  • What did you expect to happen when you ran that?
  • What options did you consider, and what made this one look right?
  • What would have helped you notice sooner?
  • Has this procedure worked before? What was different this time?

If someone's action made sense to them at the time, the fix is in whatever made it make sense: the tooling, the runbook, the naming, the dashboard. That is the practical meaning of blameless.

Step 6: Write action items with an owner, a ticket and a due date

Google's workbook lists the failure modes: no owner, vague verbs like "improve" or "make better", every item at the same priority, items that only rely on people being more careful, and items never filed as tracked bugs. Atlassian splits actions into "priority actions" (address the cause; SLO of four or eight weeks depending on the service) and "improvement actions", and tracks both in the owning team's backlog.

A good action item has a verifiable end state. "Improve backups" is not one. "Alert when the latest successful restore test is older than 7 days" is. File them from the document so the links exist from day one:

# File a priority action in the owning team's repo (GitHub CLI 2.102)
gh issue create \
  --repo acme/payments-api \
  --title "PM INC-2026-0142: validate generated feature file size before publish" \
  --label "postmortem,priority-action" \
  --assignee alice \
  --milestone "2026-11" \
  --body "Postmortem: https://github.com/acme/postmortems/blob/main/postmortems/2026-10-20-checkout-api-5xx-after-config-push.md
Done when: publish pipeline rejects files over the consumer's limit, with a test.
Due: 2026-11-17 (4-week priority action SLO)."

Keep the list short. PagerDuty's guidance is that only P0 and P1 items should normally become tickets, and that accepting some risk is fine when the fix costs more than the problem.

Step 7: Run the postmortem review meeting

Share the draft 24 hours ahead so people read it before the meeting, not during. PagerDuty's agenda fits in 15 to 30 minutes, with the incident commander chairing and the owner doing most of the talking:

  1. Walk the timeline and agree it is accurate.
  2. Review the key findings and anything surprising.
  3. Ask how this could have been caught earlier (canary, tests, load testing, alerts).
  4. Discuss customer impact and any customer feedback.
  5. Review action items: are they the right ones, owned and dated?

The chair's main job is language. When someone says "Bob should have checked", rephrase it as "what would have made the check obvious or automatic?" and move on. One correction in the first meeting sets the tone for the next fifty.

Step 8: Publish widely and track to closure

Google's chapter puts it bluntly: "an unreviewed postmortem might as well never have existed". Publish to the widest audience that can learn from it, strip anything that identifies an end user, and keep all postmortems in one searchable place. A Git repo of markdown files works well because it is greppable and versioned, and pull requests give you review and approval for free.

The workbook also flags delayed publication as an anti-pattern; its bad example came out four months after the incident. Aim for days, not weeks.

Blameless postmortem flow: from resolved incident to closed actions 1. Resolved Incident commander names one owner; review meeting booked before writing starts 2. Timeline git log, CI runs, k8s events, pager log, chat export; all UTC, facts with links, no judgements 3. Contributing factors Trigger plus conditions: detection, change, limits, recovery, people, comms 4. Action items One owner, one ticket, a due date and a verifiable end state for every item 5. Review meeting 15 to 30 min, within 3 days (SEV1) or 5 business days (SEV2); draft shared 24h ahead 6. Publish and track Share widely, no user data; priority actions due in 4 or 8 weeks, approver signs off Overdue priority action? Escalate to the service approver, not the person who was on call. Timings from PagerDuty's public response docs; action SLOs from Atlassian's incident handbook.
The process in this guide. Meeting windows follow PagerDuty's published postmortem process; the 4 or 8 week priority action SLO is Atlassian's, as described in its incident handbook.

Step 9: Copy the blameless postmortem template

Save this as postmortems/TEMPLATE.md. The {{ID}}, {{TITLE}} and {{DATE}} placeholders are filled by the script in Step 2. The sections follow Google's guidance on context, impact numbers, ownership and tracked actions.

# Postmortem: {{TITLE}}

| Field | Value |
|---|---|
| Incident ID | {{ID}} |
| Date | {{DATE}} |
| Severity | SEV? |
| Status | Draft / In review / Approved / Closed |
| Owner | @name (one person) |
| Incident commander | @name |
| Approvers | @service-owner-manager |
| Review meeting | YYYY-MM-DD HH:MM UTC |

## Summary
Two or three sentences a non-engineer can follow: what users saw, for how long, and the current state.

## Impact
- Duration: start HH:MM UTC to end HH:MM UTC (N minutes)
- Users/customers affected: number or best estimate, and how it was measured
- Error budget consumed: N% of the 30-day budget
- Data loss: none / describe
- Revenue, SLA credits, support tickets: numbers or "unknown"

## Background
What the reader needs to know about the system. Define any internal names.

## Timeline (UTC)
| Time | Source | Event | Link |
|---|---|---|---|
| 09:41 | deploy log | Config change X rolled out to 5% | link |
| 09:44 | alert | p99 latency alert fired for checkout | link |
| 09:52 | pager | On-call acknowledged | link |
| 10:20 | chat | Rollback started | link |
| 10:31 | dashboard | Error rate back to baseline | link |

## Detection
How did we find out? How long from start of impact to first alert, and to a human acknowledging it?

## Trigger
The event that started the incident, stated neutrally.

## Contributing factors
- Detection:
- Change process:
- Tooling, limits and defaults:
- Recovery and rollback:
- People, load and on-call:
- Communication:

## What went well
## What went poorly
## Where we got lucky

## Action items
| Type | Action (verifiable end state) | Owner | Ticket | Due |
|---|---|---|---|---|
| Priority | Alert when restore test is older than 7 days | @name | link | YYYY-MM-DD |
| Improvement | Add runbook step to capture k8s events | @name | link | YYYY-MM-DD |

## Supporting material
Queries and commands used, dashboards, call recording, customer communication.

Troubleshooting

Symptom Likely cause Fix
Postmortems arrive weeks late or never No owner named at incident close, meeting not booked IC names one owner on the call; book the review before drafting
Draft reads like a list of who did what wrong Timeline written from memory with judgement words Rebuild from system timestamps; strike adjectives and "should have"
Every incident has the same "human error" root cause Analysis stops at the first person in the chain Ask how the action made sense; list conditions under each category
Action items never close No tracked tickets, no due dates, no approver File tickets from the doc, set a 4 or 8 week SLO, report overdue items to approvers
Timeline has gaps or impossible ordering Mixed time zones, Kubernetes events expired after 1h Normalise to UTC; capture kubectl events during the incident
Nobody reads published postmortems Shared only within the team, buried in a wiki Publish to one searchable repo, announce in an engineering channel, review monthly
Review meeting runs over an hour Draft not read in advance, debate about fixes Share 24h ahead; take design debates offline as their own action item
Same incident repeats months later Only mitigation actions, nothing preventive Require at least one priority action that changes a system, not a behaviour

FAQ

What is the difference between a root cause and contributing factors?

The trigger or "root cause" is the single event that started the incident. Contributing factors are the conditions that let it cause damage, slowed detection or slowed recovery. Most lasting fixes come from the contributing factors, as both the Cloudflare and GitLab postmortems show.

How long should a postmortem take to write?

Aim to have a reviewed draft within the meeting window: three calendar days for a SEV-1 and five business days for a SEV-2 in PagerDuty's process. A typical document takes a few hours of the owner's time plus short interviews. Publishing months later loses most of the detail.

Should postmortems name the people involved?

Name roles and actions in the timeline where it helps the reader follow events, but never assign blame to a person or team. Google's workbook lists blaming individuals as an anti-pattern. If someone made a reasonable choice that turned out badly, the finding is about what made that choice reasonable.

Can AI write the postmortem for me?

AI tools are good at drafting a timeline from the incident channel and call transcript, which saves real time. They are not good at the contributing-factor analysis or the interviews, and they will happily produce a confident single root cause. Use them for the first draft of the timeline and check every timestamp against the source.

What I check first in any postmortem, mine included, is the action items table: does each row have one owner, a ticket link and a date, and is at least one of them a change to a system rather than a reminder to people? If that table is right, the rest of the document usually is too. When I run reviews I keep the meeting short, the language neutral and the approver in the room. Start from the Google SRE postmortem chapter, the SRE workbook's examples, PagerDuty's response docs and Atlassian's handbook. For the alerting side that feeds the timeline, see the SLO burn rate alerts guide.

Related: AIOps in 2026: 47% on ITBench, 10% on Hard On-Call, $6.50 an Investigation, and Four Outages Where the Automation Was the Incident, microVM Production Incidents: 4 Postmortems From My Fleet and Incident Management in 2026: Opsgenie Ends April 2027, AI SREs Arrive.

I'm Ajay Kumar — I build and operate PandaStack, an open-source Firecracker microVM cloud for AI agents, and write about the infrastructure and tooling around it.

Need this kind of infrastructure work? See what I do or email hello@ajayk.sh.


Related