Serialization Best Practices

Q: Should every struct use all three tag types?

Only structs that cross HTTP and SQL boundaries need json, db, and validate. Split DTOs when shapes diverge materially.

Q: How do frameworks fit these rules?

gin, echo, and chi still need explicit body limits and validation wiring. Framework binders do not replace defense in depth.

Q: Should we standardize one validator instance?

Yes - package-level validator.New() with custom registrations in init. Avoid per-request engine creation.

Team conventions for JSON tags, decode limits, validation order, and evolution tests.

Serialization bugs show up as silent data loss, not compile errors.

These rules keep wire formats stable across deploys and client versions.

How to Use This List

Tick rules as your service template adopts them.
Encode Tier A items in CI: golden JSON tests, body size limits, and validator coverage on public DTOs.
Use as a PR rubric for any change touching struct tags or API payloads.
Revisit when adding gRPC gateways, new mobile clients, or high-throughput ingest paths.

A - Tags and DTO shape

Pick one JSON naming convention per API (usually snake_case) and mirror it in struct tags. Reviewers reject mixed userId and user_id keys.
Keep json, db, and validate tags aligned on shared structs unless legacy schema forces a documented exception. Grep catches drift in review.
Use json:"-" for secrets and internal-only fields. Never log the full struct if sensitive fields exist without redaction.
Model optional numbers and booleans with pointers when zero is meaningful. Distinguish absent from explicit zero for clients.
Prefer explicit structs over map[string]any for stable public APIs. Maps hide schema from compilers and OpenAPI.

B - Decode, validate, and respond

Wrap r.Body with http.MaxBytesReader before any decode. Reject oversize payloads before allocation attacks succeed.
Decode with json.NewDecoder on request paths; set DisallowUnknownFields for strict internal/admin APIs only. Public APIs stay permissive for forward compatibility.
Validate with go-playground/validator (or equivalent) immediately after decode. Serialization is not validation.
Return stable, field-level 400 errors without leaking stack traces. Map validator.ValidationErrors to documented problem JSON.
Set Content-Type: application/json; charset=utf-8 on JSON responses. Clients and caches rely on explicit headers.

C - Custom marshaling and formats

Implement symmetric MarshalJSON and UnmarshalJSON when customizing types. Round-trip tests are mandatory.
Use the alias pattern to avoid recursive marshaling. Copy fields to type alias T inside methods.
Choose JSON for HTTP; protobuf for internal gRPC; gob only for Go-local ephemeral caches. Document format per system boundary in ADRs.
Do not persist gob blobs as long-term storage. Field renames break gob silently across deploys.
Benchmark before adopting sonic/jsoniter; pin versions and keep stdlib rollback for one release. Speed changes are not free.

D - Schema evolution and testing

Add fields; do not rename or retype JSON keys in place. Use deprecation windows and dual-key unmarshalers during migration.
Keep golden JSON fixtures per API version in testdata/. CI unmarshals old payloads into new structs.
Fuzz Unmarshal on security-sensitive DTOs. Catch panics and unexpected type combinations early.
Document null vs absent semantics for optional pointer fields. Mobile and OpenAPI clients depend on consistent behavior.
Encode integers above 2^53 as strings when JavaScript clients consume them. Prevent silent precision loss.

E - Operations and observability

Log structured fields, not raw request bodies, in production. Redact tokens and PII at the logging boundary.
Metric decode/validate failures separately from 500s. Spikes in 400 validation often signal client bugs or attacks.
Cap nested depth and collection sizes in application policy where stdlib cannot. Extra guardrails for admin-import endpoints.
Version external webhook parsers with json.RawMessage envelopes. Forward unknown event types without crashing workers.
Record serialization library choice (stdlib vs sonic) in service README. On-call needs to know behavior differences during incidents.

FAQs

Which rules are CI blockers?

Tier A: body size limits on handlers, golden JSON tests for public DTOs, and validator on create/update endpoints.

Tag naming is enforced in review unless a linter is configured.

Should every struct use all three tag types?

Only structs that cross HTTP and SQL boundaries need json, db, and validate.

Split DTOs when shapes diverge materially.

How strict should public APIs be on unknown fields?

Stay permissive by default.

Use metrics and OpenAPI diff in CI to catch client typos without breaking old apps.

When is a separate response DTO required?

When responses must hide write-only fields, join columns, or versioned shapes.

Do not reuse database models for public JSON blindly.

How do frameworks fit these rules?

gin, echo, and chi still need explicit body limits and validation wiring.

Framework binders do not replace defense in depth.

What about gRPC JSON gateways?

Protobuf owns the contract on the wire.

Apply the same evolution discipline to transcoded JSON field names documented for browsers.

Should we standardize one validator instance?

Yes - package-level validator.New() with custom registrations in init.

Avoid per-request engine creation.

How do these rules apply to workers?

Queue consumers decode JSON the same way as HTTP.

Apply size limits on message bytes from the broker client.

What goes in an ADR?

JSON naming, strict vs permissive decode policy, format per boundary, and deprecation timelines.

Link to testdata fixtures.

How do I onboard new services?

Copy the team handler template: MaxBytesReader, decode, validate, map errors.

Point engineers to Serialization Basics first.

Serialization in Go: JSON First, Formats on Demand - conceptual overview
Serialization Basics - runnable starting examples
Struct Tags for JSON, DB & Validation - tag reference
Schema Evolution & Unknown Field Handling - forward-compatible APIs
Validation with go-playground/validator - validation wiring

Stack versions: This page was written for Go 1.26.x (Green Tea GC default, go fix modernizers - verify patch at build), chi (latest - verify at build), gin (latest - verify at build), echo (latest - verify at build), google.golang.org/grpc (latest - verify at build), sigs.k8s.io/controller-runtime (latest - verify at build), kubebuilder (latest - verify at build), tinygo (latest - verify board targets at build), wazero (latest - verify at build), and golangci-lint (latest - verify linter set at build).

Serialization Best Practices

Team conventions for JSON tags, decode limits, validation order, and evolution tests.

Serialization bugs show up as silent data loss, not compile errors.

These rules keep wire formats stable across deploys and client versions.

How to Use This List

Tick rules as your service template adopts them.
Encode Tier A items in CI: golden JSON tests, body size limits, and validator coverage on public DTOs.
Use as a PR rubric for any change touching struct tags or API payloads.
Revisit when adding gRPC gateways, new mobile clients, or high-throughput ingest paths.

A - Tags and DTO shape

Pick one JSON naming convention per API (usually snake_case) and mirror it in struct tags. Reviewers reject mixed userId and user_id keys.
Keep json, db, and validate tags aligned on shared structs unless legacy schema forces a documented exception. Grep catches drift in review.
Use json:"-" for secrets and internal-only fields. Never log the full struct if sensitive fields exist without redaction.
Model optional numbers and booleans with pointers when zero is meaningful. Distinguish absent from explicit zero for clients.
Prefer explicit structs over map[string]any for stable public APIs. Maps hide schema from compilers and OpenAPI.

B - Decode, validate, and respond

Wrap r.Body with http.MaxBytesReader before any decode. Reject oversize payloads before allocation attacks succeed.
Decode with json.NewDecoder on request paths; set DisallowUnknownFields for strict internal/admin APIs only. Public APIs stay permissive for forward compatibility.
Validate with go-playground/validator (or equivalent) immediately after decode. Serialization is not validation.
Return stable, field-level 400 errors without leaking stack traces. Map validator.ValidationErrors to documented problem JSON.
Set Content-Type: application/json; charset=utf-8 on JSON responses. Clients and caches rely on explicit headers.

C - Custom marshaling and formats

Implement symmetric MarshalJSON and UnmarshalJSON when customizing types. Round-trip tests are mandatory.
Use the alias pattern to avoid recursive marshaling. Copy fields to type alias T inside methods.
Choose JSON for HTTP; protobuf for internal gRPC; gob only for Go-local ephemeral caches. Document format per system boundary in ADRs.
Do not persist gob blobs as long-term storage. Field renames break gob silently across deploys.
Benchmark before adopting sonic/jsoniter; pin versions and keep stdlib rollback for one release. Speed changes are not free.

D - Schema evolution and testing

Add fields; do not rename or retype JSON keys in place. Use deprecation windows and dual-key unmarshalers during migration.
Keep golden JSON fixtures per API version in testdata/. CI unmarshals old payloads into new structs.
Fuzz Unmarshal on security-sensitive DTOs. Catch panics and unexpected type combinations early.
Document null vs absent semantics for optional pointer fields. Mobile and OpenAPI clients depend on consistent behavior.
Encode integers above 2^53 as strings when JavaScript clients consume them. Prevent silent precision loss.

E - Operations and observability

Log structured fields, not raw request bodies, in production. Redact tokens and PII at the logging boundary.
Metric decode/validate failures separately from 500s. Spikes in 400 validation often signal client bugs or attacks.
Cap nested depth and collection sizes in application policy where stdlib cannot. Extra guardrails for admin-import endpoints.
Version external webhook parsers with json.RawMessage envelopes. Forward unknown event types without crashing workers.
Record serialization library choice (stdlib vs sonic) in service README. On-call needs to know behavior differences during incidents.

FAQs

Which rules are CI blockers?

Tier A: body size limits on handlers, golden JSON tests for public DTOs, and validator on create/update endpoints.

Tag naming is enforced in review unless a linter is configured.

Should every struct use all three tag types?

Only structs that cross HTTP and SQL boundaries need json, db, and validate.

Split DTOs when shapes diverge materially.

How strict should public APIs be on unknown fields?

Stay permissive by default.

Use metrics and OpenAPI diff in CI to catch client typos without breaking old apps.

When is a separate response DTO required?

When responses must hide write-only fields, join columns, or versioned shapes.

Do not reuse database models for public JSON blindly.

How do frameworks fit these rules?

gin, echo, and chi still need explicit body limits and validation wiring.

Framework binders do not replace defense in depth.

What about gRPC JSON gateways?

Protobuf owns the contract on the wire.

Apply the same evolution discipline to transcoded JSON field names documented for browsers.

Should we standardize one validator instance?

Yes - package-level validator.New() with custom registrations in init.

Avoid per-request engine creation.

How do these rules apply to workers?

Queue consumers decode JSON the same way as HTTP.

Apply size limits on message bytes from the broker client.

What goes in an ADR?

JSON naming, strict vs permissive decode policy, format per boundary, and deprecation timelines.

Link to testdata fixtures.

How do I onboard new services?

Copy the team handler template: MaxBytesReader, decode, validate, map errors.

Point engineers to Serialization Basics first.

Serialization in Go: JSON First, Formats on Demand - conceptual overview
Serialization Basics - runnable starting examples
Struct Tags for JSON, DB & Validation - tag reference
Schema Evolution & Unknown Field Handling - forward-compatible APIs
Validation with go-playground/validator - validation wiring

Stack versions: This page was written for Go 1.26.x (Green Tea GC default, go fix modernizers - verify patch at build), chi (latest - verify at build), gin (latest - verify at build), echo (latest - verify at build), google.golang.org/grpc (latest - verify at build), sigs.k8s.io/controller-runtime (latest - verify at build), kubebuilder (latest - verify at build), tinygo (latest - verify board targets at build), wazero (latest - verify at build), and golangci-lint (latest - verify linter set at build).

Serialization Best Practices

Search Documentation

Serialization Best Practices