Deployment Best Practices
Immutable artifacts, semver tags, and rollback runbooks.
Apply these practices in service templates, CI pipelines, and on-call runbooks so every Go deploy is traceable, reversible, and safe under load.
How to Use This List
- Tick items as your org's Go service template adopts them.
- Encode Tier A controls in CI (tests, govulncheck, image scan) and deployment gates.
- Map Tier B items to your change-management and incident response processes.
- Revisit when adding edge fleets, air-gapped environments, or new ingress controllers.
A - Build and artifact hygiene
- Build with
CGO_ENABLED=0unless a dependency requires cgo. Keeps Linux binaries portable for scratch/distroless images. - Embed version, commit, and build time via
-ldflags -X. Logs and metrics tie to exact CI output during incidents. - Store stripped production binaries and retain debug symbols separately. Supports crash symbolication without bloating runtime images.
- Run
go test -race ./...on merge queues for concurrent services. Catches data races before they surface under production load. - Run
govulncheck ./...in CI and fail on policy-violating reachable CVEs. Blocks known vulnerable dependency graphs. - Commit
go.sumand rungo mod verifyin CI. Detects module tampering at download time. - Never deploy unlabeled
latesttags to shared environments. Floating tags make rollbacks and audits unreliable.
B - Container images
- Use multi-stage Dockerfiles; copy only binary plus required CA/timezone files. Shrinks pull time and CVE surface.
- Run containers as non-root (
USER nonrootor numeric UID). Limits container breakout impact. - Pin base images by digest in production pipelines. Prevents silent base image mutation.
- Scan final image tags with Trivy, Grype, or your registry scanner. OS packages in distroless/base still need monitoring.
- Sign images with cosign (or equivalent) and verify at deploy. Supply-chain gate for promoted artifacts.
- Export SBOM alongside each release. Compliance and faster CVE impact analysis.
C - Kubernetes manifests
- Set CPU and memory requests from load-test measurements, not guesses. HPA and schedulers depend on accurate requests.
- Set memory limits from peak RSS profiles; avoid OOMKill surprises. Go heap growth is workload-dependent.
- Configure liveness on cheap
/healthz; readiness on/readyzwith dependency checks. Prevents routing to cold or broken pods. - Use
maxUnavailable: 0(or low) with readiness gates for zero-downtime rollouts. Maintains capacity during image updates. - Set
terminationGracePeriodSeconds>=http.Server.Shutdowntimeout. Drains in-flight HTTP before SIGKILL. - Define PodDisruptionBudgets for services with SLOs. Node drains respect minimum availability.
- Keep
revisionHistoryLimit>= 3 for fastkubectl rollout undo. Recent ReplicaSets remain for emergency revert.
D - CI/CD and promotion
- Build once per commit; promote the same digest through staging to production. Rebuilds at deploy time diverge bytes from tested artifacts.
- Block deploy if staging rollout or smoke tests fail.
/readyzand critical path integration tests gate promotion. - Record image digest, git SHA, and deploy actor in change tickets. Audit trail for compliance and postmortems.
- Use GitOps or manifest repos for cluster desired state. Human-reviewed infra changes separate from app code velocity.
- Automate staging deploy on main; require approval for production. Balances speed with operational control.
E - Runtime configuration and secrets
- Inject config via environment variables or mounted secrets, not baked into images. Twelve-factor parity across environments.
- Fail process start when required env vars are missing. Avoids half-configured pods passing liveness only to fail on first request.
- Mount TLS materials from cert-manager or cloud KMS integrations. No private keys in container layers.
- Separate config per environment with kustomize overlays or Helm values. Same image, different env files.
F - Observability and operations
- Expose Prometheus metrics and structured logs with request IDs. Rollouts and chaos experiments need before/after signals.
- Dashboard rollout success: error rate, p99 latency, ready replica count. Abort deploy when burn rate spikes.
- Document rollback runbook:
kubectl rollout undo, previous digest, and verification steps. On-call executes without searching Slack history. - Run game days with pod delete and latency injection in staging quarterly. Validates PDBs, probes, and client timeouts.
- Practice edge/air-gap update procedure with checksum and signature verification. Field updates match CI security posture.
G - Edge and air-gapped specifics
- Cross-compile per target
GOOS/GOARCHin CI matrix. One laptop build is not enough for ARM edge fleets. - Ship SHA-256 manifests and signatures with every field bundle. Tampered binaries never restart systemd units.
- Use atomic binary rename or stop-then-copy install scripts. Avoids executing partially written files.
- Bundle corporate CA roots when outbound HTTPS is required without public internet. Static binaries do not include trust stores automatically.
Related
- Deploying Go: Single Binary as a Superpower - why immutable binaries matter
- Deployment Basics - first build, image, and kubectl path
- CI/CD Pipelines for Go Services - pipeline template
- Probes, HPA & Pod Disruption Budgets - availability controls
- Single-Binary Edge & Air-Gapped Deployments - offline shipping patterns
Stack versions: This page was written for Go 1.26.x (Green Tea GC default, go fix modernizers - verify patch at build), chi (latest - verify at build), gin (latest - verify at build), echo (latest - verify at build), google.golang.org/grpc (latest - verify at build), sigs.k8s.io/controller-runtime (latest - verify at build), kubebuilder (latest - verify at build), tinygo (latest - verify board targets at build), wazero (latest - verify at build), and golangci-lint (latest - verify linter set at build).