database/sql Connection Pooling & Timeouts
*sql.DB is a pool of connections, not a single long-lived socket.
Tuning SetMaxOpenConns, idle settings, and lifetimes keeps your service from starving the database or wedging goroutines waiting for a free connection.
Pair pool limits with context.Context deadlines so abandoned HTTP requests release pool slots promptly.
Summary
SetMaxOpenConns caps simultaneous connections per process.
SetMaxIdleConns controls how many connections stay warm between bursts.
SetConnMaxLifetime and SetConnMaxIdleTime rotate connections for load balancer and credential hygiene.
QueryContext and friends honor cancellation while waiting for a connection or executing a query.
Recipe
Quick-reference recipe card - copy-paste ready.
func ConfigurePool(db *sql.DB) {
db.SetMaxOpenConns(25)
db.SetMaxIdleConns(10)
db.SetConnMaxLifetime(30 * time.Minute)
db.SetConnMaxIdleTime(5 * time.Minute)
}
func GetOrder(ctx context.Context, db *sql.DB, id string) (string, error) {
ctx, cancel := context.WithTimeout(ctx, 300*time.Millisecond)
defer cancel()
var status string
err := db.QueryRowContext(ctx,
`SELECT status FROM orders WHERE id = $1`, id,
).Scan(&status)
return status, err
}When to reach for this:
- Any production service using
database/sqlbehind HTTP or gRPC. - When metrics show
WaitCountrising or queries outlive client disconnect. - After scaling replicas - divide database
max_connectionsby pod count.
Working Example
package main
import (
"context"
"database/sql"
"fmt"
"time"
_ "github.com/mattn/go-sqlite3"
)
func main() {
db, err := sql.Open("sqlite3", ":memory:")
if err != nil {
panic(err)
}
defer db.Close()
db.SetMaxOpenConns(5)
db.SetMaxIdleConns(2)
db.SetConnMaxLifetime(time.Hour)
ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
defer cancel()
var one int
err = db.QueryRowContext(ctx, `SELECT 1`).Scan(&one)
fmt.Println(one, err)
s := db.Stats()
fmt.Printf("waitCount=%d waitDuration=%s\n", s.WaitCount, s.WaitDuration)
}What this demonstrates:
- Pool limits apply before the first query runs.
QueryRowContextrespects the parent deadline.db.Stats()exposesWaitCountandWaitDurationfor saturation signals.- SQLite in-memory still follows the same API as networked drivers.
Deep Dive
How It Works
sql.Opencreates a pool manager; connections are created lazily up toMaxOpenConns.- When all connections are busy, new
QueryContextcalls block until a connection frees orctxends. - Returned connections go to the idle pool if under
MaxIdleConns; otherwise they close. ConnMaxLifetimeforces replacement even if the connection is healthy, which helps behind rotating credentials or PgBouncer.
Pool Settings at a Glance
| Setting | Effect | Starting point |
|---|---|---|
SetMaxOpenConns | Hard cap per process | floor(db_max / replicas) |
SetMaxIdleConns | Warm connections kept | Often MaxOpenConns / 2 |
SetConnMaxLifetime | Max age before recycle | 15-60 minutes behind LB |
SetConnMaxIdleTime | Close idle past duration | 2-10 minutes |
PingContext | Health check at startup | Required in main |
Context vs SQL Timeouts
ctx, cancel := context.WithTimeout(parent, 200*time.Millisecond)
defer cancel()
_, err := db.ExecContext(ctx, `SET statement_timeout = 150`)- Context cancel stops waiting for pool slots and driver round trips when supported.
- Postgres
statement_timeoutprotects when drivers finish the current packet slowly. - Layer both: context for request budget, SQL for driver-agnostic guardrails.
Go Notes
- One
*sql.DBper DSN per process is typical; do not open a new pool per request. - Pass
*sql.DBfrommaininto repositories; avoidinit()pools without tests. - Log
db.Stats()fields (InUse,Idle,WaitCount) on a timer in production.
Gotchas
- Default unlimited
MaxOpenConns- A traffic spike can open thousands of connections and hit Postgrestoo many connections. Fix: set explicit cap from capacity planning. - Huge
MaxIdleConnsequal toMaxOpenConnson tiny DBs - Idle sockets waste DB slots. Fix: idle pool smaller than open cap unless latency data says otherwise. - No
Pingat startup - Bad credentials fail on first user request. Fix:PingContextwith boot timeout inmain. - Child timeout longer than handler - Query runs after client left. Fix: derive DB timeout from
r.Context()remaining deadline. - Ignoring
WaitCountmetrics - Saturation looks like slow queries. Fix: alert on wait duration growth before p99 query time spikes. - New pool per integration test without Close - CI exhausts ephemeral ports. Fix: one pool per test package or shared test container with teardown.
Alternatives
| Alternative | Use When | Don't Use When |
|---|---|---|
| PgBouncer / RDS Proxy | Many small services share one DB | You need session-level prepared stmt stickiness without transaction mode care |
statement_timeout only | Driver cancel is weak | Replacing request-scoped context entirely |
Single connection (MaxOpenConns=1) | SQLite file locks, tiny tools | Concurrent HTTP handlers |
| ORM-managed pool | Team already standardized on GORM | You need explicit db.Stats() in ops runbooks |
sql.DB per schema | Hard multi-tenant isolation | One service with one role user is enough |
FAQs
What is a good MaxOpenConns starting point?
Divide database max_connections by replica count, subtract admin and migration headroom, then cap per process.
Measure WaitCount and adjust.
Does SetConnMaxLifetime close active queries?
It prevents new uses after the deadline; in-flight work finishes unless context cancels it.
Pair with lifetimes under load balancer idle timeouts.
Should idle conns equal max open?
Not always - idle connections consume DB slots.
Start with half of max open and tune with metrics.
How do contexts interact with pool wait?
If no connection is free, QueryContext waits until ctx ends and returns context.DeadlineExceeded.
That is why handler ctx matters for pool health.
One DB per request?
No - pools are process-scoped.
Requests borrow and return connections quickly.
How do I test pool exhaustion?
Set MaxOpenConns(1), run two concurrent queries, assert the second respects a short ctx timeout.
Use httptest plus errgroup in integration tests.
Does SQLite need pool tuning?
File databases serialize writes; still set reasonable limits for in-memory test parallelism.
Networked SQL databases benefit most from tuning.
What metrics should I export?
OpenConnections, InUse, Idle, WaitCount, WaitDuration, and query latency histograms tagged by statement name.
Can gRPC and HTTP share one pool?
Yes - inject the same *sql.DB into both servers in main.
Separate pools only for different DSNs or isolation requirements.
How does this relate to context package rules?
Handler r.Context() should be the parent of every QueryContext.
Related
- Databases Basics - Open, Query, Scan fundamentals
- Data Access in Go: database/sql as the Foundation - architectural anchor
- Transactions, Prepared Statements & sql.Tx - short transactions
- context in database/sql & gRPC - cancellation wiring
- Databases & Data Access Best Practices - pool and timeout checklist
Stack versions: This page was written for Go 1.26.x (Green Tea GC default, go fix modernizers - verify patch at build), chi (latest - verify at build), gin (latest - verify at build), echo (latest - verify at build), google.golang.org/grpc (latest - verify at build), sigs.k8s.io/controller-runtime (latest - verify at build), kubebuilder (latest - verify at build), tinygo (latest - verify board targets at build), wazero (latest - verify at build), and golangci-lint (latest - verify linter set at build).