Post-Mortem Template for Go Incidents
Dense reference for blameless post-mortems on Go service outages.
//
Search across all documentation pages
Dense reference for blameless post-mortems on Go service outages.
Copy sections into your incident doc.
Collect Go-specific runtime evidence so follow-ups are actionable, not narrative-only.
| Field | Value |
|---|---|
| Incident ID | <INC-1234> |
| Title | <Short customer-facing description> |
| SEV | <1 / 2 / 3> |
| Date (UTC) | <YYYY-MM-DD> |
| Duration | <start - end> |
| Author | <name> |
| Status | <Draft / Final> |
| Prompt | Your answer |
|---|---|
| Customer impact | <who, what broke, error/latency magnitude> |
| Root cause (one line) | <technical cause in plain language> |
| Trigger | <deploy, traffic, dependency, config> |
| Fix shipped | <rollback, patch, scale, flag> |
| Recurrence risk | <low/med/high until follow-ups done> |
| Time (UTC) | Event | Actor |
|---|---|---|
<t0> | Alert fired: <alert name> | PagerDuty |
<t1> | On-call acknowledged | <name> |
<t2> | Mitigation: <rollback/scale> | <name> |
<t3> | Root cause identified | <name> |
<t4> | Incident resolved | <name> |
<t5> | Post-mortem scheduled | <name> |
| Item | Value |
|---|---|
| Go version | <runtime.Version() / image tag> |
| Build SHA | <git sha from /healthz or CI> |
GOMAXPROCS | <env or automaxprocs> |
GOMEMLIMIT | <value or unset> |
GOGC | <default 100 or override> |
| Container memory limit | <K8s limit> |
| Replica count | <at peak> |
| GC / heap metrics link | <dashboard URL> |
| Artifact | Collected? | Location |
|---|---|---|
| CPU profile (20-30s) | <Y/N> | <path or link> |
Heap profile (inuse + alloc) | <Y/N> | <path> |
Goroutine dump debug=2 | <Y/N> | <path> |
sql.DB.Stats() graph | <Y/N> | <link> |
| Trace ID exemplars | <Y/N> | <link> |
| Deploy diff / config diff | <Y/N> | <link> |
| Panic stack logs | <Y/N> | <link> |
| # | Why? |
|---|---|
| 1 | <symptom> because <immediate cause> |
| 2 | Because <deeper cause> |
| 3 | Because <process or design gap> |
| 4 | Because <missing guardrail> |
| 5 | Because <root organizational/technical gap> |
| Category | Factor |
|---|---|
| Code | <e.g. missing ctx deadline> |
| Config | <e.g. MaxOpenConns too high x replicas> |
| Observability | <e.g. no WaitCount alert> |
| Process | <e.g. deploy during freeze> |
| Dependency | <e.g. DB failover> |
<e.g. rollback runbook under 5 minutes><e.g. profiles saved before restart><e.g. pprof token expired><e.g. no GOMEMLIMIT on new service>| ID | Action | Owner | Priority | Due |
|---|---|---|---|---|
| 1 | <code fix> | <team> | P0 | <date> |
| 2 | <alert> | <team> | P1 | <date> |
| 3 | <runbook update> | <team> | P2 | <date> |
| 4 | <game day scenario> | <team> | P2 | <date> |
| Symptom family | Typical action |
|---|---|
| OOMKilled | Set GOMEMLIMIT, fix leak, right-size limit |
| GC / latency | Reduce alloc hot path, canary GOGC |
| Pool exhaustion | Resize MaxOpenConns, add WaitCount alert |
| Goroutine leak | ctx cancel, goleak CI, leak profile in staging |
| Panic | Nil interface audit, add -race test |
Two to four pages including tables.
Depth matters in timeline and follow-ups, not prose length.
Focus on systems and guardrails.
HR-sensitive conduct is out of band from the engineering doc.
Document hypotheses ruled out, evidence gaps, and monitoring added to catch the next occurrence.
Yes for SEV definitions.
Mark customer-visible vs internal clearly in executive summary.
Service owner and EM for P0/P1.
Track in the same system as normal engineering work.
Team policy varies.
Common rule: any SEV-1/2 or SLO budget burn above threshold.
Yes.
Pre-rollback profiles prove the regression; post-rollback confirms fix.
Object storage with incident ID prefix.
Link from evidence table; do not paste binary in wiki.
Always record runtime.Version().
GC behavior differs across releases, especially 1.26 Green Tea.
Use the same timeline structure.
Redact sensitive details; involve security team separately.
Stack versions: This page was written for Go 1.26.x (Green Tea GC default, go fix modernizers - verify patch at build), chi (latest - verify at build), gin (latest - verify at build), echo (latest - verify at build), google.golang.org/grpc (latest - verify at build), sigs.k8s.io/controller-runtime (latest - verify at build), kubebuilder (latest - verify at build), tinygo (latest - verify board targets at build), wazero (latest - verify at build), and golangci-lint (latest - verify linter set at build).
Reviewed by Chris St. John·Last updated Jul 18, 2026