sliceutil

sliceutil filters, maps, and reduces slices with generic functions, and summarizes numeric slices with seventeen descriptive statistics in one pass.

-
Updated:

Table of contents

Part of nurago, a collection of independent Go packages for backend services.

import "github.com/tecnickcom/nurago/pkg/sliceutil"

Filter, map and reduce over slices, plus Stats, which summarises a numeric slice with seventeen descriptive statistics in one call.

Stats

Stats takes a numeric slice and returns seventeen descriptive statistics:

data := []int{53, 83, 13, 79, 13, 37, 83, 29, 37, 13, 83, 83}

ds, err := sliceutil.Stats(data)

// ds.Count, ds.Sum, ds.Min, ds.MinID, ds.Max, ds.MaxID, ds.Range,
// ds.Mean, ds.Median, ds.Mode, ds.ModeFreq, ds.MeanDev, ds.Variance,
// ds.StdDev, ds.Skewness, ds.ExKurtosis, ds.Entropy

Most of that list is routine. Four entries are less common in a Go statistics helper:

  • MinID and MaxID give the index of the extreme value, not just the value. Finding which request was slowest usually matters more than knowing how slow it was.
  • Skewness and ExKurtosis measure asymmetry and tail weight. A mean and a standard deviation describe a bimodal latency distribution as though it were normal; these two separate the cases.
  • Entropy over the value frequencies, which is a quick way to notice that a supposedly random distribution is not.

Mode and ModeFreq come back together for a reason: ModeFreq == 1 means nothing repeated and there is no true mode, in which case Mode holds the smallest value rather than an arbitrary one. Ties break to the smallest tied value, so the result does not depend on input order.

The degenerate cases are defined rather than left to produce NaN. Skewness needs at least 3 values and a non-zero standard deviation, excess kurtosis at least 4; outside those bounds both report 0. Entropy is reported in nats, not bits, so divide by ln(2) if you want bits, and it is only meaningful for non-negative data with a positive sum. Signed data, or a total of zero, reports 0 rather than an infinity.

Two numbers keep the element type and can therefore overflow it: Sum and Range. On a []int8 of extreme values, or a very large []int32, use a wider element type.

The function clones and sorts to get the median and the mode, so it is O(n log n) and allocates a second slice the size of the first. It is a reporting and diagnostics tool, not something to call per request. An empty slice returns ErrEmptySlice rather than panicking.

The type parameter is typeutil.Number, so it accepts any integer or float width without conversion at the call site. The fields that must be fractional come back as float64; Min, Max, Sum, Mode and Range keep the element type.

Filter, Map, Reduce

The same three transforms as maputil, over slices. Go’s standard slices package has sorting, searching, comparison, Insert, Delete, Compact and the iterator helpers, but no Filter and no Map, because a Map that changes the element type cannot be expressed as a method and the language has no comprehension syntax.

As with maputil, the slice constraint is ~[]E, so a named slice type survives Filter and Reduce. Map returns []U because it changes the element type.

Worth knowing: Filter pre-allocates to len(s), trading memory for a single allocation. Filtering a million-element slice down to three entries leaves you holding capacity for a million. Copy the result, or write the loop, when that matters.

Index-aware callbacks

All three transforms pass the index as the first callback argument, not just the element:

sliceutil.Map(s, func(i int, v string) string {
    return fmt.Sprintf("%d:%s", i, v)
})

That keeps positional work, such as pairing an element with its offset or comparing against a neighbour, inside the callback rather than forcing a separate counter. Reduce passes index, element and accumulator in that order.

Example

s := []string{"Hello", "World", "Extra"}

filterFn := func(_ int, v string) bool { return v == "World" }

fmt.Println(sliceutil.Filter(s, filterFn))

// Output:
// [World]

Full source is in example_sliceutil_test.go. More runnable examples are on pkg.go.dev.

Dependencies

This package reaches no external module: it uses only the Go standard library, plus typeutil for the numeric constraint.