encode

Part of nurago, a collection of independent Go packages for backend services.

import "github.com/tecnickcom/nurago/pkg/encode"

Turns a Go value into a string safe to put in a database column, a queue message, or a cache key, and turns it back.

Two families do that here. Encode/Decode use encoding/gob wrapped in Base64. Serialize/Deserialize use encoding/json wrapped in Base64. There are also the bare Base64EncodeString, Base64DecodeString, and streaming Base64Encoder/Base64Decoder.

Measure before assuming gob is faster

Gob has a reputation as the fast binary option and JSON as the slow interoperable one. On this package’s own benchmarks, over the same value, that is backwards:

Measured with go test -bench on Go 1.27.1, linux/amd64, Intel Core i7-1260P. Run go test -bench=. ./pkg/encode/ to reproduce them on your own hardware; the absolute values will differ, the ratios much less so.

Both families encode the same value, a struct of one 43-character string, an int, a float64, and a five-element string slice.

JSON encodes 2.6 times faster and decodes 5.4 times faster, with a twelfth of the allocations on the decode path.

This follows from how gob works rather than from a defect in it. Gob is a self-describing streaming format: the first value of a given type on a stream carries a full type definition, and the decoder builds a decoding engine through reflection to match the wire type to the local type. Over a long stream of many values that cost amortises away. These helpers encode one value per call and discard the stream, so it is paid in full every time.

What each family handles

Serialize/Deserialize produce interoperable output. The payload under the Base64 is ordinary JSON, readable by any language, and the encoding covers whatever encoding/json covers: exported fields, json struct tags, and the standard marshaller interfaces.

Encode/Decode reach Go types JSON does not round-trip faithfully, and preserve them exactly: unexported-field-free structs carrying int64 beyond 2^53, maps with non-string keys, and values reaching the decoder as the same concrete types they left as.

The package’s own caveats record one security-relevant difference: gob is not designed to decode adversarial input. Its type-description mechanism is a decoder-side attack surface, and a crafted payload can drive it into large allocations. A payload that has been outside the process counts as untrusted, including one that round-tripped through a database another service can write to.

Three caveats worth reading before use

Decoding stops after one value. Trailing bytes are ignored rather than rejected. If you are appending to a stored blob and expecting the decoder to complain, it will not.

The reader- and buffer-based decoders do not bound their input. There is no size limit unless you impose one. Wrap the reader in io.LimitReader for anything untrusted, and note this applies to the JSON family too.

JSON encoding appends a trailing newline, and that newline is inside the Base64 output. Two logically identical values encoded through different paths can therefore produce different strings, which matters if you are using the encoded form as a cache key or comparing it for equality.

The Base64 layer

Base64EncodeString runs at 76 ns and Base64DecodeString at 57 ns. Both allocate, because both return a new value; that is unavoidable in the string-in, string-out shape. The streaming Base64Encoder/Base64Decoder pair exists for the cases where the payload is large enough that you would rather not hold the encoded and decoded forms in memory at the same time.

Base64 costs a third more bytes than the raw encoding, and in exchange the result survives any transport that expects text: a VARCHAR column, a JSON string field, an HTTP header.

Example

type TestData struct {
    Alpha string
    Beta  int
}

data := &TestData{Alpha: "test_string", Beta: -9876}

v, err := encode.Serialize(data)

fmt.Println(v)

// Output:
// eyJBbHBoYSI6InRlc3Rfc3RyaW5nIiwiQmV0YSI6LTk4NzZ9Cg==

The Cg== at the end is that trailing newline, Base64-encoded.

Full source is in example_encode_test.go. More runnable examples are on pkg.go.dev.

Dependencies

This package reaches no external module: it uses only the Go standard library.

Links

• Reference documentation
• Source code
• Fixed-width hex instead: uhex
• All nurago packages