stringkey

stringkey derives a stable, compact, non-cryptographic key from multiple text fields for lookup, deduplication, and idempotency-style identifiers.

Table of contents

Part of nurago, a collection of independent Go packages for backend services.

import "github.com/tecnickcom/nurago/pkg/stringkey"

Package stringkey derives a stable, compact, non-cryptographic key from multiple text fields for lookup, deduplication, and idempotency-style identifiers.

How It Works

New builds a StringKey from one or more input fields using this pipeline:

  1. Trim leading and trailing Unicode whitespace for each field.
  2. Collapse repeated Unicode whitespace to a single space.
  3. Convert text to lowercase.
  4. Concatenate fields using a tab separator (\t) to preserve boundaries.
  5. Apply Unicode NFC normalization (equivalent to NFD then NFC).
  6. Hash the resulting byte sequence with FarmHash64.

The resulting key is stored as an internal uint64 and can be retrieved in multiple representations:

  • [StringKey.Key]: raw uint64 value.
  • [StringKey.String]: base-36 string (short, variable length).
  • [StringKey.Hex]: fixed 16-character lowercase hexadecimal string.

Two normalization details are worth calling out:

  • Normalization is NFC, not NFKC. Canonically equivalent forms (such as precomposed vs decomposed accents) collapse together, but compatibility variants do not: ligatures (for example “ff”), full-width vs half-width forms, and super/subscripts are left distinct.
  • Lowercasing uses the locale-independent unicode.ToLower simple mapping, not language-tailored case folding. For example, the Turkish dotless “ı” and dotted “İ” are not special-cased.

Stability

For a fixed input, the key is fully deterministic and does not change across runs, architectures, or Go versions: FarmHash64 is a fixed algorithm and the representations are plain integer encodings.

The one dependency to be aware of is the Unicode normalization step, which uses the Unicode data tables shipped by golang.org/x/text. Keys for pure-ASCII input and for long-assigned characters are effectively permanent. Only rare or newly-assigned code points can normalize differently if the golang.org/x/text Unicode version advances, which would change their keys across such an upgrade. If you persist keys and must survive Unicode-table upgrades unchanged, pin the golang.org/x/text version alongside the stored keys.

Important Limits

This package is not cryptographically secure and must not be used for security tokens, signatures, or untrusted collision-resistant identifiers.

It is designed for reasonably small input sizes and a moderate number of keys. According to the birthday bound for a 64-bit hash space (~1.8x10^19):

  • collision probability is about 1% around 6.1x10^8 generated keys;
  • collision probability is about 50% around 5.1x10^9 generated keys.

Choose a larger hash or cryptographic construction when your scale or threat model requires stronger collision guarantees.

When To Use

  • A composite lookup key or deduplication key must be built from multiple fields.
  • The key needs to be stable across processes and releases.

Example

// input strings
args := []string{
	"0123456789",
	"abcdefghijklmnopqrstuvwxyz",
	"Lorem ipsum dolor sit amet",
}

// generate a new key
sk := stringkey.New(args...)

fmt.Println(sk)

// Output:
// 2p8dmari397l8

Full source is in example_stringkey_test.go. More runnable examples are on pkg.go.dev.

Dependencies

Importing this package pulls 2 external modules:

  • github.com/tecnickcom/farmhash64
  • golang.org/x/text