prometheus

prometheus implements the nurago metrics contract on the Prometheus client ecosystem, with runtime, HTTP, SQL and error collectors registered by default.

-
Updated:

Table of contents

Part of nurago, a collection of independent Go packages for backend services.

import "github.com/tecnickcom/nurago/pkg/metrics/prometheus"

A metrics.Client backed by prometheus/client_golang.

m, err := prometheus.New()

What you get without defining a metric

The default registration is the reason to use this rather than wiring client_golang yourself. Out of New you have:

  • Go runtime and process collectors: goroutine count, heap, GC pause, open file descriptors, CPU.
  • Inbound HTTP: request count, in-flight gauge, duration histogram, request size histogram, response size histogram.
  • Outbound HTTP: request count, in-flight gauge, duration histogram.
  • SQL connection pool stats, through dlmiddlecote/sqlstats.
  • Error counters by log level and by task, operation and code.

That set answers most of what a service dashboard asks, and defining it by hand is a few hundred lines that every service repeats.

The SQL pool metrics are the ones teams most often lack and most need. sql.DB maintains a pool whose saturation is invisible until requests start queueing; WaitCount and WaitDuration show exhaustion directly, where latency alone looks like a slow database.

Pull, not push

Prometheus scrapes. MetricsHandlerFunc returns the handler to mount, normally at /metrics:

router.HandlerFunc(http.MethodGet, "/metrics", m.MetricsHandlerFunc())

Two consequences. The endpoint must be reachable by the scraper and should not be public, so it belongs on an internal listener or behind network policy alongside profiling. And metrics are only as fresh as the scrape interval, so a short-lived process can finish before it is ever scraped. For batch jobs and CLI tools, statsd suits better.

Histogram buckets have to be set explicitly

Prometheus histograms have fixed bucket boundaries set at definition time, and the quantiles you can compute later depend entirely on them. Defaults tuned for a sub-second API produce useless percentiles for a service whose p99 is eight seconds: every observation lands in +Inf.

The options exist for that:

prometheus.WithInboundRequestSizeBuckets(buckets)

with matching options for the other histograms. Set them from the latency and size range your service actually has. Changing them later invalidates historical comparison, so it is worth measuring before choosing.

WithCollector registers a collector of your own for domain metrics alongside the defaults. WithHandlerOpts passes promhttp.HandlerOpts through, for error handling and compression behaviour on the scrape endpoint.

Cardinality

InstrumentHandler(path, handler) takes the route pattern, not the request URL. That is what keeps cardinality bounded: /users/:id is one label value, where /users/12345 would be one per user and would eventually take down the Prometheus server.

Registering handlers with the literal path is the standard way to cause that, and it is worth checking in review. The same applies to IncErrorCounter: task, operation and code must come from a bounded set, never from user input or an error string.

Dependencies

Reaches github.com/prometheus/client_golang and github.com/dlmiddlecote/sqlstats.