navree

File storage

Named disks on object_store: putting, reading and streaming files, upload checks and signed temporary URLs.

On this page

Featurestorage navree-storage gives your app named disks: places to keep files, each with its own root. You write and read files by path on a disk, stream large files without loading them into memory, accept uploads with a size limit and a content check, and hand out links that expire. It is built on object_store, the storage layer of Apache Arrow.

Configuration

Add StorageModule to your app. With no configuration you get one disk, local, rooted at storage/app (relative to the working directory):

src/app.rs

Disks are described in the storage section. Each disk has a driver; local disks also take a root and an optional public_url:

config/app.toml
KeyDefaultWhat it does
storage.defaultlocalThe disk injected as Disk and returned by Storage::default_disk(). It must exist.
storage.disks.<name>.driverlocal or memory.
storage.disks.<name>.rootThe directory of a local disk. It is created at boot if missing.
storage.disks.<name>.public_urlnoneWhere a local disk's root is publicly served (a CDN or a static file server), for Disk::url.
storage.base_urlnoneAn absolute http(s) URL put in front of temporary URLs. Without it they are path-only (/storage/...).

Defining storage.disks replaces the default local disk, so list every disk you need. Disk names use letters, digits, - and _. Invalid options (a missing default disk, an empty root, a relative base_url) fail the boot with a message naming the key. Like every section, these keys can come from the environment: APP_STORAGE__BASE_URL=https://api.example.com.

You can also configure the module in code with StorageModule::for_root. Build StorageOptions from its default so you keep the local disk:

src/app.rs

StorageModule is global: every module can inject what it provides without importing it.

Getting a disk

Inject Arc<Disk> for the default disk, or Arc<Storage> to pick disks by name:

src/modules/reports/service.rs

Storage::disk(name) returns Err(StorageError::UnknownDisk) for a name that isn't configured, and Storage::disk_names() lists the configured ones. A Disk is cheap to clone.

Storing and reading files

Paths are relative to the disk's root and use / as the separator. Every method normalises them first: \ also counts as a separator, and leading, trailing and repeated separators and . segments are dropped, so /a//./b.txt is a/b.txt. A path containing .., a segment made only of dots or spaces, a control character, or more than 1024 bytes fails with StorageError::InvalidPath before it reaches the driver, so no path can leave the disk's root.

Rust
  • put replaces an existing file, and the write is atomic: readers see the old file or the new one, never half of it.
  • delete succeeds when the file doesn't exist.
  • list("") lists the whole disk.
  • get and size fail with StorageError::NotFound for a missing file.

There is no copy, move or "last modified" helper yet; read and write the file instead.

Errors in handlers

A StorageError that reaches a handler's ? turns into a response: a missing file is a 404, an invalid path a 400 with the message Invalid file path., and anything else a 500 (logged, never shown). See Errors for the response format.

Streaming large files

get and put hold the whole file in memory. For large files, stream instead. get_stream returns a ByteStream of chunks (local files are read on Tokio's blocking pool), and put_stream writes any stream of Result<Bytes, E> chunks and returns the number of bytes written:

Rust

put_stream uploads the file in parts and only makes it visible when the stream ends successfully. If the source stream fails, nothing is stored and the error is StorageError::Stream.

To send a stored file in a response, wrap the stream in an Axum body:

src/modules/avatars/controller.rs

Set the Content-Type yourself when you serve files this way; the temporary URL route does it for you.

File uploads

store_upload streams one multipart file field to a disk, checking it as it goes. Take Axum's Multipart extractor (it must be the last argument), get the field, and pass it with UploadRules:

src/modules/avatars/controller.rs

The file is stored under the directory you pass (users) with a random 40-character name and an extension that matches its content. StoredFile tells you where it went:

200 OK

original_name is the name the browser sent, without directories. It is untrusted: show it, but never use it as a path.

What gets checked

  • The type comes from the content, not from the client. The first 8 KiB are sniffed with infer. A renamed executable is not an image. Content infer doesn't recognise is text/* when it is UTF-8 text (the file name's extension picks the text/ type, text/plain otherwise) and application/octet-stream otherwise, so an SVG or HTML file is never accepted as an image.
  • Allowed types are exact (image/png) or wildcards (image/*, */*), case-insensitive. An empty list accepts any type.
  • The size limit is checked while streaming, so an oversized file is never stored.
ProblemResponse
An empty file, or a type the rules don't allow400 validation error keyed by the field name, e.g. avatar must be a file of type: image/png, image/jpeg, image/webp
A file over UploadRules::max_bytes413 avatar must not be larger than 1048576 bytes (the partial file is discarded)
A request body over the route's body limit413
A malformed multipart body400
A storage failure500

StorageModule also maps multipart errors you propagate with ? (as in form.next_field().await?) to their client status, so a body over the limit is a 413, not a 500.

Public URLs

A local disk served by a CDN or a static file server gets a public_url. Disk::url joins it with the percent-encoded path:

Rust

A disk without a public_url (memory disks always) fails with StorageError::NoPublicUrl. Navree doesn't serve the disk's root itself: point your web server or CDN at it.

Temporary URLs

temporary_url gives a link that downloads one file until a given time, with no sign-in needed. It works for every disk, private ones included:

Rust

The signature is an HMAC-SHA-256 of the disk, the path and the expiry time. With storage.base_url set, the link is absolute (https://api.example.com/storage/...).

StorageModule serves these links at GET /storage/{disk}/{*path} when your app has HttpModule. The route is public (the signature is the authorisation) and is not under the API prefix. It:

  • answers 403 Invalid signature. for a forged or altered link, an unknown disk or an invalid path, and 403 The link has expired. once the time has passed (it uses the app's dyn Clock when one is registered);
  • answers 404 when the file is gone;
  • streams the file with a Content-Type guessed from its extension, Content-Length, X-Content-Type-Options: nosniff and Cache-Control: private;
  • sends HTML, XHTML, SVG, XML and JavaScript files as attachments (Content-Disposition: attachment), so a stored file can never run in your app's origin.

Anyone holding a link can download the file until it expires, so keep expiry times short and don't log the links.

Health check

StorageModule registers a storage readiness check. On every disk it writes, reads back and deletes a small probe file under .navree-health/, and reports the disk as down if any step fails. It shows up in GET /health/ready; see Health and metrics.

Testing

Use a memory disk in tests: nothing touches the file system and every app starts empty.

src/modules/reports/tests/mod.rs

To test uploads, send a multipart/form-data body with TestClient (.header("content-type", "multipart/form-data; boundary=...") and .body(...)). Outside an app, Storage::new(&options, signing_key) builds the disks directly.

Popular pages

  • Installation
  • Your first app
  • Routing and controllers
  • Validation
  • Authentication
  • Benchmarks