sfcache

sfcache is a generic fixed-size cache with single-flight deduplication, TTL expiry and an optional stale-if-error window that survives an upstream outage.

-
Updated:

Table of contents

Part of nurago, a collection of independent Go packages for backend services.

import "github.com/tecnickcom/nurago/pkg/sfcache"

Caches the result of an expensive lookup, collapses concurrent requests for the same key into one call, and can keep serving the last good value while the upstream is down.

cache := sfcache.New(func(ctx context.Context, key string) (*Customer, error) {
    return fetchCustomer(ctx, key)
}, sfcache.Config{Size: 256, TTL: 5 * time.Minute})

customer, err := cache.Lookup(ctx, "customer:123")

The value type is inferred from the lookup function, so Lookup returns *Customer with no type assertion.

Single-flight is the part that saves you

A plain TTL cache has a failure mode that only appears under load. When a popular key expires, every request that arrives before the first refresh completes sees a miss and calls the upstream. A key fetched a thousand times a second produces a thousand simultaneous upstream calls at the moment of expiry, which is how a cache turns a slow dependency into a dead one.

Here, one goroutine performs the lookup and the rest wait for its result. The upstream sees one call.

That property is also the reason a TTL of zero or less is still useful: nothing is served from cache, but concurrent callers still coalesce. It becomes a pure deduplicator, for an idempotent but expensive operation whose result should not be cached at all.

Error handling is specified precisely

The semantics repay reading, because they differ from what most caches do.

Only successful lookups are cached. A nil value counts as a value and is cached.

Errors are shared, never cached. Callers coalesced onto the same lookup all receive the error, and the next call retries. A failing key leaves an already-expired entry behind until it is reclaimed or overwritten.

A lookup that fails with its own context’s error publishes nothing at all. This is the subtle one. If the goroutine elected to do the work has its request cancelled, its failure is not attributed to the other waiters, whose contexts are still live. Without that rule, one client disconnecting would fail every other request coalesced behind it.

Whatever the lookup returns is passed through as-is, including a non-nil value alongside a non-nil error.

Stale-if-error

With the stale window enabled, a failed refresh serves the last known good value instead of an error.

This is the difference between a dependency outage degrading your service and taking it down. A profile service that is unreachable for ninety seconds returns slightly old profiles rather than 500s, and most callers never notice.

The window is anchored to the failure, not to the original write, so the staleness budget starts when things go wrong rather than having been consumed already by normal TTL expiry.

The trade is explicit: during an outage you are serving data you know is out of date, for a bounded period you choose. For a permission or entitlement check that is the wrong behaviour, and the window should be off.

Bounded size

Config.Size caps the number of entries, with eviction when it is reached. That is what keeps an unbounded key space, such as one derived from user input, from becoming an unbounded memory leak.

Len, Reset, Remove and PurgeExpired are available for inspection and explicit management. PurgeExpired returns how many entries it reclaimed.

Per-entry TTL

WithTTLFunc computes a TTL from the key and the value rather than using one cache-wide setting.

Two uses. Honouring an upstream Cache-Control or expiry field, so your cache does not outlive what the source considers valid. And varying by result: caching a “not found” for thirty seconds while caching a found record for an hour, which avoids a negative result being pinned for the full TTL.

Dependencies

This package reaches no external module: it uses only the Go standard library.