Build, Test & Profile from the Shell

Q: Should I run go test ./... or per-package during development?

Per-package while iterating; full ./... before push and in CI. Use -run TestName to narrow further.

Q: Can I profile memory leaks from the shell?

Yes. Use -memprofile, or curl .../debug/pprof/heap and go tool pprof -top on the heap profile.

Q: How do I compare two profiles?

go tool pprof -base=old.prof new.prof highlights deltas after an optimization branch.

Q: Does go test -json help shell pipelines?

Yes. Tools like gotestsum and custom jq filters consume JSON events for CI annotations.

Q: How do I profile a short-lived CLI?

Set GODEBUG=gctrace=1 or use test benchmarks wrapping the hot path; CLIs often profile better in unit tests than via HTTP pprof.

Run builds and tests from the shell, capture profiles with go tool pprof, and line up failures with journalctl timestamps on Linux hosts.

Summary

The Go toolchain is CLI-native: go build, go test, and go tool pprof are designed for scripts, CI, and SSH sessions.

Pair them with shell redirection, tee, and journal queries so a failing test on your laptop uses the same commands you will run during a production incident.

Recipe

Quick-reference recipe card - copy-paste ready.

# Build and test from module root
go build -o bin/api ./cmd/api
go test ./... -count=1 -race 2>&1 | tee test.log
 
# CPU profile a benchmark locally
go test -bench=. -benchmem -cpuprofile=cpu.prof -run=^$ ./...
go tool pprof -http=:0 cpu.prof
 
# Fetch profile from a running server (when pprof endpoint is enabled)
curl -s "http://localhost:6060/debug/pprof/profile?seconds=30" -o live.prof
go tool pprof live.prof
 
# Correlate with systemd logs
sudo journalctl -u api.service --since "10 min ago" --no-pager

When to reach for this:

You need a repeatable build/test loop before opening a PR.
Latency regressed and you want CPU or heap evidence, not guesses.
A service failed on a VM and you must align go test output with journal timestamps.

Working Example

#!/usr/bin/env bash
set -euo pipefail
cd "$(git rev-parse --show-toplevel 2>/dev/null || pwd)"
 
# Build
mkdir -p bin
go build -ldflags="-s -w" -o bin/api ./cmd/api
 
# Test with race detector; keep log artifact
go test ./... -count=1 -race -timeout=5m 2>&1 | tee artifacts/test-$(date +%Y%m%d%H%M).log
 
# Micro-benchmark + CPU profile for one package
PKG=./internal/handler
go test "$PKG" -bench=BenchmarkHealth -benchmem \
  -cpuprofile=artifacts/cpu.prof -run=^$ -count=1
 
go tool pprof -top artifacts/cpu.prof | head -20
 
# If api.service runs on this host, pull recent journal lines
if systemctl is-active --quiet api.service 2>/dev/null; then
  journalctl -u api.service --since "15 min ago" -n 200 --no-pager \
    | tee artifacts/api.journal.snippet
fi

What this demonstrates:

Artifact directory for binaries, test logs, and profiles with timestamps.
Race-enabled go test ./... suitable for CI parity.
Benchmark-scoped CPU profile instead of profiling the entire module blindly.
journalctl snippet captured beside test output for incident notebooks.

Deep Dive

How It Works

go build compiles packages; -o names the output binary and -ldflags injects version metadata or strips debug symbols.
go test executes _test.go files; flags like -race, -count=1, and -timeout map directly to CI YAML.
go test -cpuprofile and -memprofile attach runtime/pprof sampling during tests or benchmarks.
go tool pprof reads profile files or URLs and can open a local web UI (-http) for flame graphs.
Live services expose /debug/pprof/* when you import net/http/pprof and listen on a debug port (guard in production).
journalctl reads the systemd journal; filtering by unit and time range places Go stack traces next to OOM or restart events.

Useful go test Flags

Flag	Purpose
`./...`	All packages under module root
`-race`	Race detector (CI for pure Go packages)
`-count=1`	Disable test cache
`-run=^$`	Skip tests; benchmarks only
`-bench=.`	Run benchmarks matching regex
`-cpuprofile=file`	Write CPU profile
`-memprofile=file`	Write heap profile
`-json`	Machine-readable test events (Go 1.20+)

pprof Fetch Patterns

Source	Command
Test/bench	`go test -cpuprofile=cpu.prof`
Local server	`curl localhost:6060/debug/pprof/profile?seconds=30`
K8s pod	`kubectl port-forward pod/api 6060:6060` then curl
File analysis	`go tool pprof -http=:0 cpu.prof`

Go Notes

// cmd/api/main.go - optional debug listener (staging only)
import _ "net/http/pprof"
 
go func() {
    log.Println(http.ListenAndServe("127.0.0.1:6060", nil))
}()

Bind to loopback, protect with auth or network policy before exposing beyond localhost.

Gotchas

Profiling without load - A 30s CPU profile on an idle server shows idle stacks. Fix: reproduce traffic with hey, k6, or integration tests while profiling.
Leaving pprof on :6060 public - Attackers scan for open pprof ports. Fix: bind 127.0.0.1, use port-forward, or gate with auth.
Test cache hiding failures - Flaky tests pass locally after a green run. Fix: go test -count=1 in CI and when debugging flakes.
Race detector only on pure Go - CGO-heavy packages may need tagged builds. Fix: split race runs per package or use -tags consistently.
Huge go test ./... logs - Failures scroll off screen in tmux. Fix: always tee to a file and rg FAIL afterward.
Journal timezone confusion - journalctl uses local time by default; CI UTC differs. Fix: pass --utc or note offsets in incident docs.

Alternatives

Alternative	Use When	Don't Use When
IDE test runner	Fast inner loop while editing	You need the exact CI command transcript
`make test` wrapper	Team-standard flags and artifacts	It hides flags new hires never learn
Continuous profiling (Parca, Pyroscope)	Always-on production history	You only need a one-off local benchmark
`perf` / `bpftrace`	Kernel-level or syscall issues	Go CPU hotspots are usually enough in user space
Cloud vendor profiler	Managed GCP/AWS integration	Air-gapped or minimal dependency policy

FAQs

Should I run go test ./... or per-package during development?

Per-package while iterating; full ./... before push and in CI.

Use -run TestName to narrow further.

Where does go test write profile files?

Relative to the package directory under test unless you pass an absolute path.

Collect them into artifacts/ in scripts to avoid dirtying git trees.

How long should a CPU profile sample run?

15-30 seconds is typical for HTTP services under steady load.

Shorter samples work for tight benchmarks.

Can I profile memory leaks from the shell?

Yes.

Use -memprofile, or curl .../debug/pprof/heap and go tool pprof -top on the heap profile.

What does journalctl add over tail -f app.log?

Kernel messages, systemd restart reasons, and unit metadata on the same timeline as your Go logs when stdout goes to the journal.

Should CI always use -race?

Strong default for services without heavy CGO.

Budget extra time and skip only with documented exceptions.

How do I compare two profiles?

go tool pprof -base=old.prof new.prof highlights deltas after an optimization branch.

Does go test -json help shell pipelines?

Yes.

Tools like gotestsum and custom jq filters consume JSON events for CI annotations.

Why strip symbols with -ldflags="-s -w"?

Smaller binaries for container images.

Keep unstripped builds for internal crash symbolication if you lack separate debug info.

How do I profile a short-lived CLI?

Set GODEBUG=gctrace=1 or use test benchmarks wrapping the hot path; CLIs often profile better in unit tests than via HTTP pprof.

What if pprof says profile is empty?

The process likely had no samples during the window.

Increase load or extend seconds= query parameter.

Can journalctl filter by Go panic strings?

journalctl -u api.service -g panic searches message text.

Export a window to a file and rg for stack frames as a backup.

Linux CLI Basics - tee and log piping
Search, ripgrep & jq for Debugging - search test and log output
systemd & Service Unit Files for Go Binaries - journal integration
Containers from the CLI: docker & kubectl - port-forward for pprof
Go Testing Basics - Go test depth

Stack versions: This page was written for Go 1.26.x (Green Tea GC default, go fix modernizers - verify patch at build), chi (latest - verify at build), gin (latest - verify at build), echo (latest - verify at build), google.golang.org/grpc (latest - verify at build), sigs.k8s.io/controller-runtime (latest - verify at build), kubebuilder (latest - verify at build), tinygo (latest - verify board targets at build), wazero (latest - verify at build), and golangci-lint (latest - verify linter set at build).

Summary

The Go toolchain is CLI-native: go build, go test, and go tool pprof are designed for scripts, CI, and SSH sessions.

Pair them with shell redirection, tee, and journal queries so a failing test on your laptop uses the same commands you will run during a production incident.

Recipe

Quick-reference recipe card - copy-paste ready.

# Build and test from module root
go build -o bin/api ./cmd/api
go test ./... -count=1 -race 2>&1 | tee test.log
 
# CPU profile a benchmark locally
go test -bench=. -benchmem -cpuprofile=cpu.prof -run=^$ ./...
go tool pprof -http=:0 cpu.prof
 
# Fetch profile from a running server (when pprof endpoint is enabled)
curl -s "http://localhost:6060/debug/pprof/profile?seconds=30" -o live.prof
go tool pprof live.prof
 
# Correlate with systemd logs
sudo journalctl -u api.service --since "10 min ago" --no-pager

When to reach for this:

You need a repeatable build/test loop before opening a PR.
Latency regressed and you want CPU or heap evidence, not guesses.
A service failed on a VM and you must align go test output with journal timestamps.

Working Example

#!/usr/bin/env bash
set -euo pipefail
cd "$(git rev-parse --show-toplevel 2>/dev/null || pwd)"
 
# Build
mkdir -p bin
go build -ldflags="-s -w" -o bin/api ./cmd/api
 
# Test with race detector; keep log artifact
go test ./... -count=1 -race -timeout=5m 2>&1 | tee artifacts/test-$(date +%Y%m%d%H%M).log
 
# Micro-benchmark + CPU profile for one package
PKG=./internal/handler
go test "$PKG" -bench=BenchmarkHealth -benchmem \
  -cpuprofile=artifacts/cpu.prof -run=^$ -count=1
 
go tool pprof -top artifacts/cpu.prof | head -20
 
# If api.service runs on this host, pull recent journal lines
if systemctl is-active --quiet api.service 2>/dev/null; then
  journalctl -u api.service --since "15 min ago" -n 200 --no-pager \
    | tee artifacts/api.journal.snippet
fi

What this demonstrates:

Artifact directory for binaries, test logs, and profiles with timestamps.
Race-enabled go test ./... suitable for CI parity.
Benchmark-scoped CPU profile instead of profiling the entire module blindly.
journalctl snippet captured beside test output for incident notebooks.

Deep Dive

How It Works

go build compiles packages; -o names the output binary and -ldflags injects version metadata or strips debug symbols.
go test executes _test.go files; flags like -race, -count=1, and -timeout map directly to CI YAML.
go test -cpuprofile and -memprofile attach runtime/pprof sampling during tests or benchmarks.
go tool pprof reads profile files or URLs and can open a local web UI (-http) for flame graphs.
Live services expose /debug/pprof/* when you import net/http/pprof and listen on a debug port (guard in production).
journalctl reads the systemd journal; filtering by unit and time range places Go stack traces next to OOM or restart events.

Useful go test Flags

Flag	Purpose
`./...`	All packages under module root
`-race`	Race detector (CI for pure Go packages)
`-count=1`	Disable test cache
`-run=^$`	Skip tests; benchmarks only
`-bench=.`	Run benchmarks matching regex
`-cpuprofile=file`	Write CPU profile
`-memprofile=file`	Write heap profile
`-json`	Machine-readable test events (Go 1.20+)

pprof Fetch Patterns

Source	Command
Test/bench	`go test -cpuprofile=cpu.prof`
Local server	`curl localhost:6060/debug/pprof/profile?seconds=30`
K8s pod	`kubectl port-forward pod/api 6060:6060` then curl
File analysis	`go tool pprof -http=:0 cpu.prof`

Go Notes

// cmd/api/main.go - optional debug listener (staging only)
import _ "net/http/pprof"
 
go func() {
    log.Println(http.ListenAndServe("127.0.0.1:6060", nil))
}()

Bind to loopback, protect with auth or network policy before exposing beyond localhost.

Gotchas

Profiling without load - A 30s CPU profile on an idle server shows idle stacks. Fix: reproduce traffic with hey, k6, or integration tests while profiling.
Leaving pprof on :6060 public - Attackers scan for open pprof ports. Fix: bind 127.0.0.1, use port-forward, or gate with auth.
Test cache hiding failures - Flaky tests pass locally after a green run. Fix: go test -count=1 in CI and when debugging flakes.
Race detector only on pure Go - CGO-heavy packages may need tagged builds. Fix: split race runs per package or use -tags consistently.
Huge go test ./... logs - Failures scroll off screen in tmux. Fix: always tee to a file and rg FAIL afterward.
Journal timezone confusion - journalctl uses local time by default; CI UTC differs. Fix: pass --utc or note offsets in incident docs.

Alternatives

Alternative	Use When	Don't Use When
IDE test runner	Fast inner loop while editing	You need the exact CI command transcript
`make test` wrapper	Team-standard flags and artifacts	It hides flags new hires never learn
Continuous profiling (Parca, Pyroscope)	Always-on production history	You only need a one-off local benchmark
`perf` / `bpftrace`	Kernel-level or syscall issues	Go CPU hotspots are usually enough in user space
Cloud vendor profiler	Managed GCP/AWS integration	Air-gapped or minimal dependency policy

FAQs

Should I run go test ./... or per-package during development?

Per-package while iterating; full ./... before push and in CI.

Use -run TestName to narrow further.

Where does go test write profile files?

Relative to the package directory under test unless you pass an absolute path.

Collect them into artifacts/ in scripts to avoid dirtying git trees.

How long should a CPU profile sample run?

15-30 seconds is typical for HTTP services under steady load.

Shorter samples work for tight benchmarks.

Can I profile memory leaks from the shell?

Yes.

Use -memprofile, or curl .../debug/pprof/heap and go tool pprof -top on the heap profile.

What does journalctl add over tail -f app.log?

Kernel messages, systemd restart reasons, and unit metadata on the same timeline as your Go logs when stdout goes to the journal.

Should CI always use -race?

Strong default for services without heavy CGO.

Budget extra time and skip only with documented exceptions.

How do I compare two profiles?

go tool pprof -base=old.prof new.prof highlights deltas after an optimization branch.

Does go test -json help shell pipelines?

Yes.

Tools like gotestsum and custom jq filters consume JSON events for CI annotations.

Why strip symbols with -ldflags="-s -w"?

Smaller binaries for container images.

Keep unstripped builds for internal crash symbolication if you lack separate debug info.

How do I profile a short-lived CLI?

Set GODEBUG=gctrace=1 or use test benchmarks wrapping the hot path; CLIs often profile better in unit tests than via HTTP pprof.

What if pprof says profile is empty?

The process likely had no samples during the window.

Increase load or extend seconds= query parameter.

Can journalctl filter by Go panic strings?

journalctl -u api.service -g panic searches message text.

Export a window to a file and rg for stack frames as a backup.

Linux CLI Basics - tee and log piping
Search, ripgrep & jq for Debugging - search test and log output
systemd & Service Unit Files for Go Binaries - journal integration
Containers from the CLI: docker & kubectl - port-forward for pprof
Go Testing Basics - Go test depth

Stack versions: This page was written for Go 1.26.x (Green Tea GC default, go fix modernizers - verify patch at build), chi (latest - verify at build), gin (latest - verify at build), echo (latest - verify at build), google.golang.org/grpc (latest - verify at build), sigs.k8s.io/controller-runtime (latest - verify at build), kubebuilder (latest - verify at build), tinygo (latest - verify board targets at build), wazero (latest - verify at build), and golangci-lint (latest - verify linter set at build).

Build, Test & Profile from the Shell

Search Documentation

Build, Test & Profile from the Shell