Overview

storage is a standalone Go module (github.com/gp-system/storage): object storage for uploads, avatars, generated reports, exports, anything that isn't a database row. In the shop sample app, the admin surface uploads product images through it. The root module has zero gp-system dependencies, and only three third-party ones (go.opentelemetry.io/otel, .../metric, .../trace), so pulling it into a project that doesn't otherwise use the kit costs nothing beyond OTel's own API package. It works in any Go program: a CLI tool, a Lambda, a plain net/http service, with or without the gpsystem kit installed.

Install

go get github.com/gp-system/storage@v0.1.0 # the kit itself uses this tag
go get github.com/gp-system/storage@latest

Go 1.23+ (the Files iterator on Disk uses range-over-func). The built-in drivers are separate imports: storage/memory and storage/local ship inside the same module, at no extra dependency cost. The S3-compatible driver is different, see S3 for why it's its own module.

The model: Driver, Disk, Manager

The package is deliberately layered into three pieces, each with one job:

Driver: the backend contract

type Driver interface {
    Name() string
    Put(ctx context.Context, path string, r io.Reader, opts PutOptions) error
    Reader(ctx context.Context, path string) (io.ReadCloser, error)
    RangeReader(ctx context.Context, path string, offset, length int64) (io.ReadCloser, error)
    Stat(ctx context.Context, path string) (FileInfo, error)
    List(ctx context.Context, opts ListOptions) (ListPage, error)
    Copy(ctx context.Context, src, dst string) error
    Delete(ctx context.Context, path string) error
    URL(ctx context.Context, path string) (string, error)
    TemporaryURL(ctx context.Context, path string, opts TemporaryURLOptions) (string, error)
    TemporaryUploadURL(ctx context.Context, path string, opts TemporaryUploadURLOptions) (UploadURL, error)
}

A Driver doesn't have to support everything: a driver that can't produce signed URLs returns ErrUnsupported from TemporaryURL/TemporaryUploadURL rather than faking one, and callers that care check for it (see Disk.ProvidesTemporaryURLs). A driver may additionally implement the optional Mover interface:

type Mover interface {
    Move(ctx context.Context, src, dst string) error
}

if it can move an object natively (a rename, not a byte-for-byte copy); Disk.Move uses it when available and falls back to Copy + Delete otherwise.

Three built-in drivers ship with the ecosystem: storage/memory (in-process, for tests), storage/local (filesystem, for single-server deployments), and storage/driver/s3 (S3-compatible, for production). All three satisfy the same Driver contract, and storage/storagetest runs one conformance suite against all of them, plus any driver you write yourself.

ValidPath

func ValidPath(path string) bool

A small guard every Driver implementation is expected to call before touching the backend: it rejects the empty path, absolute paths, ./.. segments, and anything else that could escape the storage root. Writing a custom driver means calling it at the top of Put, Copy, and friends, and returning ErrInvalidPath when it's false.

Disk: the API you actually call

disk := storage.NewDisk(driver, storage.WithName("uploads"))

NewDisk wraps a Driver with a name and OTel instrumentation (a span per operation, with the disk name and path as attributes), and is what service code depends on day to day; see Disk API for the full method set. Options: WithName(string), WithTracerProvider, WithMeterProvider; with none of the latter two set, instrumentation is a no-op, same as everywhere else in the kit.

Manager: more than one named disk

mgr := storage.New("public", map[string]*storage.Disk{
    "public":  storage.NewDisk(local.New(cfg), storage.WithName("public")),
    "backups": storage.NewDisk(s3.New(s3cfg), storage.WithName("backups")),
})

mgr.Default().Put(ctx, "avatars/42.png", r, storage.PutOptions{})
backups, _ := mgr.Disk("backups")

A single Disk is all most projects need. Manager exists for the case where one app genuinely uses more than one: public assets on local disk, nightly database dumps on S3, say. New(defaultDiskName, disks) builds it; Default() returns the named default, Disk(name) looks one up by name (with an ok bool the same shape as a map lookup), and Names() lists everything registered.

Decorators: composing behavior onto a Driver

Both wrap any Driver and return another Driver, so they compose:

d := storage.ReadOnly(s3.New(cfg))                    // rejects writes
scoped, err := storage.Scoped(s3.New(cfg), "tenant-42") // confines to a prefix

• ReadOnly(Driver) Driver: every Put, Delete, and Copy call returns ErrReadOnly immediately, without reaching the backend. Useful for a disk that serves archived or externally-managed content, where a bug that tries to write should fail loudly and locally, not silently succeed against production storage.
• Scoped(Driver, prefix string) (Driver, error): confines every operation to paths under prefix, transparently. A common use is per-tenant isolation on a shared bucket: give each tenant a Disk built from Scoped(sharedDriver, "tenants/"+tenantID), and a bug that assembles the wrong path for one tenant can't reach another tenant's files, because the scoped driver never sees paths outside its prefix in the first place.

Both compose with each other and with the built-in drivers freely, since a Driver wrapping a Driver is still just a Driver.

Related pages

• Disk API: the full method set: writes, reads, listing, URLs.
• Memory and local disk: the two drivers that ship in the root module.
• S3: the S3-compatible driver, its own Go module.
• Testing: the conformance suite, and writing a custom driver.
• Architecture: the three deliberate exceptions: why storage is one of the few places the kit hides a dependency behind an interface.