filter

Part of nurago, a collection of independent Go packages for backend services.

import "github.com/tecnickcom/nurago/pkg/filter"

Filters a slice against structured rules supplied by the caller, or parsed from JSON or a query parameter.

Rules are [][]Rule, with the outer slice AND and the inner slice OR:

[[A], [B, C], [D]]   evaluates as   A AND (B OR C) AND D

Built for untrusted rules

A filter expression that arrives from a client is code someone else wrote, running in your process. The package treats it that way, and the bounds are the substance of the design.

Each closes a specific amplification. Without a rule count, one request carries ten thousand rules and every one is evaluated per element. Without a value length, a regular expression operand is unbounded. Without a field depth, a deeply nested path costs reflection work per element.

Go’s regexp is RE2, so it has no catastrophic backtracking, which removes the worst regular-expression denial of service by construction. The remaining costs are linear and bounded by the options above.

Malformed against unmatched

The distinction is careful and worth understanding, because it decides what becomes a 400 and what becomes an empty result.

A field selector that cannot be resolved against the element type is a client error: it names no such field, descends into a non-struct, targets an unexported field, or exceeds the depth limit. These are rejected with ErrInvalidFilter before any element is touched, because the answer is the same for every element and an early rejection avoids the work entirely.

A field that is merely unreachable on a given element makes that element a non-match rather than an error. That covers a nil pointer along the path, and a field absent from the concrete type of an element in a []any, both of which are knowable only per element.

A pointer or interface leaf is dereferenced; a nil leaf compares as a nil operand.

Structural problems are rejected too: an empty group, a rule missing any of field, type or value.

Filtering happens in place

Apply and ApplySubset take a pointer to a slice and modify it: matching elements are compacted to the front and the slice is shortened.

No second slice is allocated, so memory stays flat when filtering a large result set. Two consequences to know about: the caller’s slice is changed, so keep a copy if the original order or contents are needed; and the backing array is not zeroed past the new length, so references beyond it stay alive until the slice is replaced.

ApplySubset adds offset and length for pagination, and returns the total match count, so a paged API can report totals without a second pass.

Selectors, and matching on tags

Dot paths select nested fields. By default they match Go field names case-sensitively, so Address.Country.

WithFieldNameTag matches a struct tag instead, so a client sends address.country and never learns your Go field names. For a public API that is the right setting: it decouples the wire contract from the struct, so renaming a field is not a breaking change, and it avoids leaking internal naming.

Measured

Measured with go test -bench on Go 1.27.1, linux/amd64, Intel Core i7-1260P. Run go test -bench=. ./pkg/filter/ to reproduce them on your own hardware; the absolute values will differ, the ratios much less so.

Equality filtering scales linearly: ten times the elements costs about ten times the time, with the allocation count staying nearly flat, which confirms the in-place filtering is not allocating per element.

Regular-expression matching costs 4.4 times equality on the same input. Tag-based field matching costs 13% over Go field names, which is a small price for the decoupling.

Reflection paths are cached per type, so repeated evaluation against the same element type does not re-resolve the selector.

Dependencies

This package reaches no external module: it uses only the Go standard library.

Links

• Reference documentation
• Source code
• In depth: Filtering Untrusted Client Queries in Go: Threat Model First
• Pagination arithmetic: paging
• All nurago packages