s3

s3 covers bucket-scoped object upload, download, listing and deletion through the AWS SDK v2, streaming through io.Reader on both directions.

-
Updated:

Table of contents

Part of nurago, a collection of independent Go packages for backend services.

import "github.com/tecnickcom/nurago/pkg/s3"

Object operations against one bucket.

c, err := s3.New(ctx, "my-bucket")

err = c.Put(ctx, "path/to/key", reader)
obj, err := c.Get(ctx, "path/to/key")
keys, err := c.ListKeys(ctx, "prefix/")
err = c.Delete(ctx, "path/to/key")

Bucket-scoped, and streaming

The Client is bound to a bucket at construction, so the bucket name is not repeated at every call site and cannot be got wrong on one of them. A service using two buckets constructs two clients, which also keeps their configurations independently visible.

Put takes an io.Reader and Get exposes the object’s body as a stream. Neither requires the object in memory.

That matters for the obvious reason: a 2 GB object uploaded from a []byte needs 2 GB of RAM. Streaming means the memory cost is the buffer size rather than the object size, and a file, an HTTP request body, or a gzip.Reader can be passed straight through.

The consequence on the read side is that Get returns something you must close. An unclosed body holds the HTTP connection out of the pool until finalisation, and enough of them exhaust it. errutil’s JoinFnError is built for reporting a deferred close failure without losing the primary error.

ListKeys against ListObjects

Two listing methods, differing in what they return.

ListKeys gives the key strings alone. ListObjects gives per-object metadata: size, last-modified time, and ETag.

The distinction is about response size rather than convenience. Listing a prefix with a hundred thousand objects, ListKeys materialises strings, ListObjects materialises a struct per object. When the keys are all you need, the lighter call is meaningfully lighter.

Both filter by prefix. S3 is a flat key space with no directories, so a prefix is the only hierarchy available, and ListKeys(ctx, "2026/09/") is what a directory listing looks like.

Prefixes and request rate

S3 partitions by key prefix, and read and write rates scale per partition. A key scheme with a common leading prefix, such as a date, concentrates a workload onto one partition and becomes a throughput ceiling under sustained load.

Where objects are written at high rates, a high-entropy leading component distributes them. uhex or stringkey produce one cheaply. For ordinary volumes this does not arise.

Health check

Client.HealthCheck satisfies the healthcheck contract directly, so it registers with no adapter and uses the same credentials and configuration as real traffic.

Configuration

AWS configuration comes from awsopt, shared with sqs and awssecretcache, so region, credentials and endpoint resolution are identical across every AWS client in the service.

Dependencies

Reaches github.com/aws/aws-sdk-go-v2/service/s3, plus awsopt.