Production Concurrency: Patterns Beyond Hello Goroutine
A go func() in a tutorial proves the runtime works.
A production service must bound parallelism, propagate cancellation, shed load under pressure, and shut down without leaving goroutines blocked forever.
This page is the conceptual anchor for Advanced Concurrency.
Advanced Concurrency Basics collects runnable snippets; sibling articles cover worker pools, pipelines, rate limiting, errgroup, atomics, leak prevention, and review habits.
Summary
- Production Go services orchestrate concurrent work through bounded worker pools, staged pipelines, context-driven cancellation, and explicit shutdown - not unbounded goroutine sprawl.
- Insight: Unbounded concurrency exhausts memory, file descriptors, and downstream quotas; missing cancellation turns slow clients into resource leaks; deadlocks in channel graphs stall entire processes.
- Key Concepts: worker pool, pipeline stage, fan-out/fan-in, backpressure, errgroup, context cancellation, graceful shutdown.
- When to Use: HTTP gateways fanning out to dependencies, batch processors, stream consumers, gRPC servers, and any service where request rate exceeds safe parallel calls to a database or API.
- Limitations/Trade-offs: More structure means more code; channels are not always clearer than mutexes; over-batching increases latency; distributed systems still need timeouts at every hop.
- Related Topics: context deadlines, semaphores, atomic counters, race detector, observability metrics.
Foundations
Think of a Go service as a traffic controller for work.
Requests arrive faster than a database can answer if every request spawns its own unbounded goroutine tree.
Production code introduces capacity: a fixed number of workers, a semaphore limiting in-flight calls, or a buffered channel that blocks producers when consumers fall behind.
Cancellation is the second pillar.
context.Context threads a stop signal from the HTTP handler through every downstream call.
When the client disconnects or a deadline fires, in-flight work should exit promptly instead of holding connections open.
The third pillar is composition.
Simple patterns combine into larger systems:
- Worker pool - N goroutines pull jobs from a queue.
- Pipeline - stage A produces, stage B transforms, stage C sinks results.
- Fan-out/fan-in - duplicate work across workers, merge results through one channel or errgroup.
These are not competing religions.
A single handler might use a semaphore for outbound HTTP, errgroup for parallel fetches, and a worker pool for background retries.
Mechanics & Interactions
Client request
|
v
HTTP handler (ctx from r.Context())
|
+----+----+
| |
v v
errgroup semaphore-limited
parallel downstream calls
fetches
| |
+----+----+
v
merge / respond
|
on ctx.Done() -> stop spawning, drain or abandon
Worker pools cap CPU-bound or I/O-bound parallelism at a number you choose - often tied to connection pool size or CPU count.
Jobs enter a channel; workers range over it until the channel closes or context cancels.
Pipelines separate concerns: parsing, validation, enrichment, and persistence can each run in its own stage with bounded buffers between them.
Backpressure appears naturally when an upstream stage blocks on send because the downstream buffer is full.
errgroup coordinates a batch of goroutines with shared cancellation: the first error cancels siblings and returns one combined error to the caller.
Pair it with context.WithCancel derived from the request context.
Rate limiting (token bucket, golang.org/x/time/rate, or a weighted semaphore) protects shared resources.
Graceful shutdown listens for SIGTERM, stops accepting new work, waits for in-flight requests with a timeout, then closes worker channels so goroutines exit.
| Pattern | Strength | Weakness | Best Fit |
|---|---|---|---|
Unbounded go per task | Simplest code | No backpressure; OOM risk | Prototypes only |
| Worker pool | Predictable parallelism | Queue sizing tuning | CPU/IO pools, job processors |
| Pipeline stages | Clear dataflow | More moving parts | ETL, log processing |
| errgroup | First-error cancel | Less control over partial results | Parallel fetches |
| Semaphore / rate limit | Protects downstream | Adds latency under load | DB/API gateways |
Advanced Considerations & Applications
HTTP servers in net/http already run each request in its own goroutine.
The advanced work is inside the handler: limit concurrent calls to Postgres, Redis, or a partner API.
Match pool size to SetMaxOpenConns on database/sql so you do not queue goroutines behind an exhausted pool.
For streaming RPC or WebSockets, one long-lived goroutine per connection is normal.
Still bound auxiliary work (background refresh, fan-out enrichment) with semaphores.
Observability closes the loop: track goroutine count (runtime.NumGoroutine), in-flight requests, queue depth, and time blocked on semaphores.
Spikes without traffic growth often signal a leak or deadlock.
Run load tests with -race in CI for packages that mutate shared state.
Kubernetes sends SIGTERM before SIGKILL.
Your main should call http.Server.Shutdown with a deadline, cancel a root context, close job channels, and Wait() on worker groups.
Document ownership: who closes which channel, who waits on whom.
Integration with context is mandatory for production handlers.
Pass r.Context() into http.NewRequestWithContext, gRPC calls, and db.QueryContext.
See context Package for the cancellation model.
Common Misconceptions
- "More goroutines always means more throughput" - Past the point where CPU, connections, or locks saturate, extra goroutines only add scheduling overhead and memory.
- "Channels replace all mutexes" - Shared caches, metrics structs, and connection registries often need
sync.Mutexor atomics; channels are for passing ownership, not every shared byte. - "errgroup replaces WaitGroup everywhere" -
sync.WaitGroupis still right when errors are handled per-goroutine and you do not need automatic cancel-on-error. - "Closing a channel stops workers" - Workers must select on
ctx.Done()as well; close alone does not unblock sends waiting on a full buffer. - "Production concurrency is only for big teams" - A single-binary API with one dependency benefits from a semaphore on day one.
FAQs
What is the first production pattern to add after learning goroutines?
A bounded semaphore or small worker pool on outbound I/O.
It is the smallest change that prevents unbounded goroutines when traffic spikes.
How do worker pools relate to GOMAXPROCS?
GOMAXPROCS caps OS threads running Go code.
Worker pools cap application-level tasks - often lower than CPU count for I/O or matching external connection limits.
When should I use channels versus a mutex for a task queue?
Channels when producers and consumers are goroutines passing work items.
A mutex plus slice when you need indexed priority, inspection, or single-goroutine drain logic.
Does net/http already give me a worker pool?
It gives one goroutine per request, not a pool for your internal work.
You still bound parallel database or HTTP client calls inside handlers.
What is backpressure in Go terms?
A slow consumer causes a blocking send on a channel or a failed Acquire on a semaphore, which slows producers instead of buffering unbounded work in memory.
How does errgroup differ from WaitGroup?
errgroup returns the first error and can cancel a derived context.
WaitGroup only waits; error handling stays manual.
Should background jobs use the request context?
No - use a separate context cancelled on process shutdown.
Request context ends when the client disconnects, which is wrong for fire-and-forget audit logs unless you explicitly detach.
How do I test concurrent shutdown?
Start workers, cancel context, close inputs, assert Wait() returns within a timeout and goroutine count returns to baseline.
Is fan-out always faster?
Only when work is independent and downstream capacity supports N parallel calls.
Otherwise fan-out amplifies throttling and errors.
What metrics signal a goroutine leak?
Monotonically rising runtime.NumGoroutine after load stops, plus growing heap and open FD counts.
Can I mix pipelines and errgroup in one handler?
Yes - a pipeline stage can use errgroup internally for parallel steps within one stage.
Where does the race detector fit?
Run go test -race on packages with shared mutable state.
It does not detect deadlocks or logical races on channels.
Related
- Advanced Concurrency Basics - runnable intro examples
- Worker Pools & Task Queues - bounded parallelism
- Fan-In, Fan-Out & Pipeline Stages - staged dataflow
- errgroup & golang.org/x/sync Extensions - coordinated goroutines
- Avoiding Goroutine Leaks & Channel Deadlocks - shutdown cheatsheet
- Concurrency in Go: Goroutines and Channels First - CSP foundations
Stack versions: This page was written for Go 1.26.x (Green Tea GC default, go fix modernizers - verify patch at build), chi (latest - verify at build), gin (latest - verify at build), echo (latest - verify at build), google.golang.org/grpc (latest - verify at build), sigs.k8s.io/controller-runtime (latest - verify at build), kubebuilder (latest - verify at build), tinygo (latest - verify board targets at build), wazero (latest - verify at build), and golangci-lint (latest - verify linter set at build).