Serialization in Go: JSON First, Formats on Demand
Go services spend most of their serialization time turning structs into JSON and back.
The standard library ships purpose-built encoders for common formats, and a small set of interfaces lets you override behavior when defaults are wrong.
Serialization Basics collects runnable snippets; sibling pages cover custom marshaling, alternate formats, struct tags, schema evolution, performance libraries, validation, and team conventions.
Summary
- Serialization converts in-memory Go values into bytes (or text) for storage and transport.
encoding/jsonis the default choice; other packages inencoding/*and third-party codecs fill specialized needs. - Insight: APIs, databases, caches, and message queues all exchange serialized payloads. Consistent tags and round-trip tests prevent silent data loss when fields rename, types change, or clients lag behind servers.
- Key Concepts: Marshal, Unmarshal, struct tag, Encoder/Decoder, MarshalJSON, json.RawMessage, omitempty, DisallowUnknownFields.
- When to Use: JSON for HTTP and human-readable config; protobuf or gob for internal RPC; XML for legacy integrations; custom marshalers when time formats, enums, or sensitive fields need special handling.
- Limitations/Trade-offs: JSON is verbose and slow compared to binary formats; zero values and pointers behave subtly; validation is separate from encoding; high-performance JSON libraries trade stdlib compatibility for speed.
- Related Topics: HTTP handlers, ORM column tags, protobuf schemas, API versioning, request size limits, fuzz testing.
Foundations
Serialization answers one question: how do you turn a User struct into bytes another program can read?
Go's answer starts with reflection-driven encoders in the standard library.
json.Marshal walks exported struct fields, reads json tags, and emits UTF-8 JSON.
json.Unmarshal does the reverse, allocating nested structs and slices as needed.
The pattern repeats across formats:
| Package | Typical use | Wire shape |
|---|---|---|
encoding/json | REST APIs, config files | Text JSON |
encoding/xml | SOAP, RSS, legacy enterprise | Text XML |
encoding/gob | Go-to-Go RPC or caches | Binary, Go-specific |
google.golang.org/protobuf | gRPC, cross-language contracts | Binary protobuf |
Most production Go code paths touch JSON first.
chi, gin, and echo handlers bind JSON bodies through encoding/json or thin wrappers.
google.golang.org/grpc uses protobuf, not JSON, on the wire (gateways may transcode to JSON).
Mechanics & Interactions
Struct tags are the primary configuration surface:
type Profile struct {
ID string `json:"id"`
Email string `json:"email,omitempty"`
Internal string `json:"-"`
CreatedAt time.Time `json:"created_at"`
}- Exported fields serialize; unexported fields are ignored.
omitemptydrops zero values from output.-hides a field from JSON entirely.
When tags are not enough, implement json.Marshaler and json.Unmarshaler:
func (t TimeOnly) MarshalJSON() ([]byte, error) {
return json.Marshal(t.Format("15:04:05"))
}The encoder checks for these interfaces before using default rules.
Streaming APIs (json.NewEncoder, json.NewDecoder) suit HTTP response bodies and large inputs without loading everything into memory.
Decoder options matter in production:
Decoder.DisallowUnknownFields()rejects unexpected keys (strict APIs).UseNumber()keeps large integers asjson.Numberinstead offloat64.
Validation belongs in a separate step.
go-playground/validator reads validate tags after JSON lands in a struct.
Serialization does not enforce business rules; it only translates shapes.
Advanced Considerations & Applications
Schema evolution is a contract problem.
Servers should add fields without breaking old clients; clients should ignore unknown keys.
Protobuf formalizes field numbers; JSON relies on discipline and tests.
json.RawMessage defers parsing nested blobs so you can forward-compatible-decode partial payloads.
Performance becomes relevant at high QPS or large batch jobs.
json-iterator and sonic (verify versions at build) mirror encoding/json APIs with faster reflection or JIT paths.
Migration risks include subtle behavioral differences in omitempty, float64 handling, and MarshalJSON ordering.
| Approach | Strength | Weakness | Best Fit |
|---|---|---|---|
encoding/json | Stable, no deps | Slower on huge payloads | Public HTTP APIs |
| Protobuf + gRPC | Compact, versioned schema | Needs codegen and tooling | Internal microservices |
encoding/gob | Simple Go-native binary | Not cross-language | Process-local caches |
encoding/xml | Enterprise interop | Verbose, fragile | Legacy feeds |
Custom MarshalJSON | Full control | Easy to get wrong | Dates, enums, redaction |
Size limits belong at the HTTP layer (http.MaxBytesReader) and in application policy, not inside json.Marshal.
golangci-lint can flag struct tags and JSON tags that drift from field names when teams enable tag-check linters.
Common Misconceptions
- "Unmarshal validates input" - It only maps JSON to Go types. Malformed business data still unmarshals successfully unless you add validation.
- "omitempty omits nil pointers only" - It omits any zero value:
"",0,false,nilpointers, empty slices, and zero times. - "json tags replace db tags" - ORMs and serializers read different tag keys; one struct often carries
json,db, andvalidatetags side by side. - "gob replaces JSON for APIs" - gob is Go-specific and unsuitable for browser or polyglot clients.
- "Custom marshalers run automatically for embedded types" - Promotion rules differ; embedded structs can shadow interfaces unless you compose carefully.
- "Faster JSON libraries are drop-in with zero tests" - Benchmark and round-trip test before switching; edge cases differ.
FAQs
Why is JSON the default in Go services?
JSON is human-readable, debuggable with curl, and supported by every client stack.
The stdlib encoder is good enough for most API latency budgets.
When should I reach for protobuf instead?
Use protobuf for internal gRPC services that need compact binary payloads and generated cross-language types.
Keep JSON at the edge for browsers and public REST.
Do I need both MarshalJSON and UnmarshalJSON?
Implement both when the type round-trips through APIs.
One-sided implementations confuse readers and break symmetry tests.
How do frameworks bind JSON bodies?
gin uses ShouldBindJSON; echo uses Bind; chi pairs with json.NewDecoder(r.Body).
All ultimately rely on encoding/json or compatible decoders.
What about YAML or TOML for config?
Use gopkg.in/yaml.v3 or github.com/pelletier/go-toml/v2 for config files.
Keep HTTP APIs on JSON unless clients require otherwise.
Can I serialize unexported fields?
Not with the stdlib reflect-based encoders.
Use custom marshalers or export fields intended for wire formats.
How do I redact secrets from JSON logs?
Use json:"-", custom MarshalJSON, or a dedicated log DTO without sensitive fields.
Never rely on omitempty for passwords.
Does TinyGo support encoding/json?
Mostly yes, but reflection limits apply on small boards.
Verify your target with the stack footer note at build time.
Should APIs use camelCase or snake_case JSON keys?
Pick one convention per API surface and encode it in struct tags.
Document the choice in your OpenAPI or README.
Where do size limits belong?
Wrap r.Body with http.MaxBytesReader before decode.
Reject oversize payloads before Unmarshal allocates large slices.
Related
- Serialization Basics - runnable Marshal/Unmarshal examples
- encoding/json Custom Marshaling - MarshalJSON patterns
- Struct Tags for JSON, DB & Validation - multi-tag conventions
- Schema Evolution & Unknown Field Handling - forward-compatible APIs
- Validation with go-playground/validator - post-decode validation
- Serialization Best Practices - team checklist
Stack versions: This page was written for Go 1.26.x (Green Tea GC default, go fix modernizers - verify patch at build), chi (latest - verify at build), gin (latest - verify at build), echo (latest - verify at build), google.golang.org/grpc (latest - verify at build), sigs.k8s.io/controller-runtime (latest - verify at build), kubebuilder (latest - verify at build), tinygo (latest - verify board targets at build), wazero (latest - verify at build), and golangci-lint (latest - verify linter set at build).