s3

s3 uploads, downloads, lists, and deletes S3 bucket objects through the AWS SDK v2 S3 client.

Table of contents

Part of nurago, a collection of independent Go packages for backend services.

import "github.com/tecnickcom/nurago/pkg/s3"

Package s3 uploads, downloads, lists, and deletes S3 bucket objects through the AWS SDK v2 S3 client.

It covers the common object operations:

  • upload object data,
  • download object data,
  • list object keys by prefix,
  • delete objects.

It is built on github.com/aws/aws-sdk-go-v2/service/s3.

What It Provides

  • New to create a bucket-scoped Client.
  • Client.Put to upload from an io.Reader.
  • Client.Get to fetch an object and access its body stream.
  • Client.ListKeys to list object keys, optionally filtered by prefix.
  • Client.ListObjects to list objects with per-object metadata (size, last-modified, ETag), optionally filtered by prefix.
  • Client.Delete to remove an object by key.
  • Client.HealthCheck to verify bucket reachability and access permissions.

Configuration & Extensibility

The client configuration composes with awsopt and exposes option hooks:

  • WithAWSOptions to pass generic AWS config options,
  • WithSrvOptionFuncs to customize S3 service options,
  • WithS3Client to inject a custom S3 implementation (tests and advanced integrations; skips AWS configuration loading),
  • WithEndpointMutable and WithEndpointImmutable for endpoint overrides (useful for local S3-compatible environments and tests).

Usage

c, err := s3.New(ctx, "my-bucket")
if err != nil {
    return err
}

if err := c.Put(ctx, "reports/latest.json", reader); err != nil {
    return err
}

obj, err := c.Get(ctx, "reports/latest.json")
if err != nil {
    return err
}
_ = obj

keys, err := c.ListKeys(ctx, "reports/")
if err != nil {
    return err
}
_ = keys

if err := c.Delete(ctx, "reports/old.json"); err != nil {
    return err
}

if err := c.HealthCheck(ctx); err != nil {
    return err
}

When To Use

  • A service puts, gets, lists, and deletes objects and nothing more exotic.
  • S3 should be reported on the service health endpoint.

Example

// A caller would normally configure a real client via options such as
// WithAWSOptions or WithEndpointMutable; here an injected client keeps the
// example self-contained, so no AWS configuration is loaded.
c, err := s3.New(
	context.TODO(),
	"my-bucket",
	s3.WithS3Client(&exampleS3Client{objects: map[string][]byte{}}),
)
if err != nil {
	fmt.Println("error:", err)

	return
}

err = c.Put(context.TODO(), "reports/latest.json", strings.NewReader(`{"ok":true}`))
if err != nil {
	fmt.Println("error:", err)

	return
}

obj, err := c.Get(context.TODO(), "reports/latest.json")
if err != nil {
	fmt.Println("error:", err)

	return
}

data, err := io.ReadAll(obj.Body())
if err != nil {
	fmt.Println("error:", err)

	return
}

err = obj.Close()
if err != nil {
	fmt.Println("error:", err)

	return
}

fmt.Println(obj.ContentType())
fmt.Println(obj.ContentLength())
fmt.Println(string(data))

keys, err := c.ListKeys(context.TODO(), "reports/")
if err != nil {
	fmt.Println("error:", err)

	return
}

fmt.Println(keys)

err = c.Delete(context.TODO(), "reports/latest.json")
if err != nil {
	fmt.Println("error:", err)

	return
}

// Output:
// application/json
// 11
// {"ok":true}
// [reports/latest.json]

Full source is in example_s3_test.go. More runnable examples are on pkg.go.dev.

Dependencies

Importing this package pulls 18 external modules:

  • github.com/aws/aws-sdk-go-v2
  • github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream
  • github.com/aws/aws-sdk-go-v2/config
  • github.com/aws/aws-sdk-go-v2/credentials
  • github.com/aws/aws-sdk-go-v2/feature/ec2/imds
  • github.com/aws/aws-sdk-go-v2/internal/configsources
  • github.com/aws/aws-sdk-go-v2/internal/endpoints/v2
  • github.com/aws/aws-sdk-go-v2/internal/v4a
  • github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding
  • github.com/aws/aws-sdk-go-v2/service/internal/checksum
  • github.com/aws/aws-sdk-go-v2/service/internal/presigned-url
  • github.com/aws/aws-sdk-go-v2/service/internal/s3shared
  • github.com/aws/aws-sdk-go-v2/service/s3
  • github.com/aws/aws-sdk-go-v2/service/signin
  • github.com/aws/aws-sdk-go-v2/service/sso
  • github.com/aws/aws-sdk-go-v2/service/ssooidc
  • github.com/aws/aws-sdk-go-v2/service/sts
  • github.com/aws/smithy-go