navree
Documentation

Inbound mail

Receive mail: messages parsed with every header, body and attachment, sources for Amazon SES, Mailgun, Postmark, raw MIME and watched Microsoft 365 and Gmail mailboxes, mailboxes routed by address, signed reply addresses and the raw .eml kept on a disk.

On this page

Introduction

Featuremail-inbound Sending mail is half of the conversation. People answer a ticket notification, reply to an announcement, forward a suspicious message to a reporting address, or write to a support inbox. Inbound mail turns each of those messages into a typed value your code can act on:

PartWhat it does
Mailboxesm.mailbox("reply+{token}@in.acme.test", handler) routes a message to a handler by the address it was sent to.
The messageInboundMail: headers, addresses, the text and HTML bodies, attachments and forwarded messages, decoded from any charset, with the raw bytes kept.
Reply tokensSigned reply addresses, so an answer finds its ticket and nobody can forge one for another.
Raw messages on a diskEvery message kept as an .eml file on a storage disk, once.
SourcesRoutes for Amazon SES (SNS and S3 actions), Mailgun routes, Postmark inbound streams and raw MIME from your own server.
Microsoft 365 and Gmail mailboxesA customer's own mailboxes, watched through their integration connection.
The eventInboundMailReceived for every message, for listeners that don't own a mailbox.

Every source hands its message to the same pipeline, so a mailbox handler doesn't know or care which provider delivered it.

Installation

Turn on the mail-inbound feature of the navree facade. It is part of the default full feature, and turns on mail and storage. Parsing runs on mail-parser, in pure Rust.

Cargo.toml

Inbound mail is part of MailModule, which the app already has to send mail. Add the modules the parts you use need:

src/app.rs

Only MailModule (and the ViewModule it always needs) is required. Without EventsModule no event is emitted; without an application key reply tokens fail with InboundError::MissingKey; mail.inbound.disk and mail.inbound.ses.disk need StorageModule, and the boot fails without it.

Inbound mail adds no table. Messages live where your handlers put them, and the raw bytes on a disk when you ask for that.

Mailboxes

A mailbox is an address pattern and a handler. Register one from any module's register, without importing anything but the extension trait (the prelude brings it):

src/modules/support/mod.rs

A handler is any closure that takes the InboundMail and a MailboxContext and returns a future of Result<(), BoxError>. The context gives the app (ctx.get::<T>(), ctx.app()), so a closure can reach any provider.

Patterns

PatternMatches
[email protected]That address. Letters compare without case.
reply+{token}@in.acme.testreply+ followed by one or more characters, captured as token, at in.acme.test.
{user}@{tenant}.in.acme.testAny address on a subdomain, with both parts captured.
*@in.acme.testAny address at in.acme.test. * matches any run of characters except @.
*Every address: a catch-all.

A {name} capture never crosses the @. Read captures with ctx.param("token"). An invalid pattern (no @, an unclosed {, a name used twice) fails the boot, naming the module and the pattern.

Mailboxes are tried most specific first: exact addresses, then patterns with captures, then patterns with *, each group in registration order. For each mailbox, the message's recipients are tried in the order of mail.recipients():

  1. the envelope recipients the provider reports (they include Bcc and forwarding addresses the headers never show);
  2. Delivered-To and X-Original-To;
  3. To, then Cc.

The first mailbox that matches one of them takes the message. It is handled once, by one mailbox.

Handlers with dependencies

A mailbox that needs services is an #[injectable] type that implements MailboxHandler. Register it with mailbox_with, which also registers it as a provider of the module:

src/modules/tickets/mailbox.rs

The handler is resolved from its module, with that module's visibility, in a request scope of its own for each message.

What becomes of a message

The handler returnsStatusThe source answersThe provider
Ok(())handled200is done.
Err(ctx.reject("reason")), or a bad reply token from ctx.verified(..)?rejected200is done: sending again would fail again.
Any other errornone500delivers the message again later.
(no mailbox matched)unrouted200is done.

Because a failure makes the provider retry, a handler may see the same message twice. Make handlers idempotent: mail.message_id and mail.fingerprint (the SHA-256 of the raw bytes) identify a message, and a unique index on either in your table turns a second delivery into a no-op.

MailboxContext has:

MethodReturns
pattern()The mailbox pattern that matched.
address()The recipient that matched it, in lower case.
param(name)A {name} capture.
verified(name), verified_uuid(name)The value sealed in the reply token captured as {name}. A bad or expired token is an error that rejects the message when returned with ?.
tokens()The app's ReplyTokens.
get::<T>(), app()Providers of the app.
reject(reason)An error that rejects the message for good.

The message

InboundMail is a parsed message. Encoded words in headers (=?UTF-8?B?...?=), quoted-printable and base64 bodies and legacy charsets (ISO-8859-1, Windows-1252, ISO-2022-JP and the rest) are decoded to UTF-8.

FieldHolds
idThis receipt's id (a UUID v7).
sourceWho delivered it: ses, mailgun, postmark, raw, graph, gmail, or test.
provider_idThe provider's id for the message (SES's messageId, Postmark's MessageID, the Graph or Gmail message id).
envelopesender and recipients of the SMTP envelope, when the provider reports them.
verdictsThe provider's spam, virus, spf, dkim and dmarc verdicts, when it reports them.
message_id, in_reply_to, referencesMessage ids without angle brackets.
from, reply_to, to, ccAddress { name, email } values; groups are flattened.
subject, dateThe decoded subject and the date in UTC.
text, htmlThe plain-text body (converted from the HTML when the message has no text part) and the HTML body.
headersEvery header in order: Header { name, value }, unfolded but not decoded.
attachmentsFiles, inline parts and forwarded messages.
size, fingerprintThe raw size in bytes and its SHA-256 in hex.
storedWhere the raw message is kept (StoredMail { disk, path }), once it is.
received_atWhen it arrived.

And these methods:

Rust

is_auto_reply() matters whenever your app answers mail: replying to an out-of-office message that replies to yours is how mail loops start. reply_text() keeps the text above On ... wrote:, -----Original Message-----, a From: block or the first > quote, which is what a ticket wants from a reply.

Attachments and forwarded messages

Each Attachment has a filename, a lower-case content_type, a content_id (without angle brackets, for inline images an HTML body references as cid:...), inline, and the decoded content as Bytes.

A message forwarded as an attachment (message/rfc822, what "Forward as attachment" and most "Report phishing" buttons send) is an attachment too. is_message() tells it apart and as_mail() parses it, headers and all:

src/modules/security/reported.rs

Register it like any closure: m.mailbox("[email protected]", reported_phish).

Verdicts

Amazon SES reports all five verdicts of its receipt rule; Mailgun its spam flag, SPF and DKIM results; Postmark its spam status and SPF result. Each is a Verdict: Pass, Fail, Gray, ProcessingFailed or Disabled. For spam and virus, Fail means the provider flagged the message; verdicts.is_flagged() is true when either did. Verdicts are only ever read from the provider's own payload, never from the message's headers, which the sender controls.

Reply tokens

Notifications that people answer need a reply address that leads back to the right record. ReplyTokens seals a value (a ticket id, announcement:42) into a token that only your app can make:

src/modules/tickets/notifications.rs

The person's answer arrives at [email protected], and the mailbox above reads the ticket back with ctx.verified_uuid("token").

  • A token is the value, the day it was issued and an 80-bit HMAC-SHA-256 tag, in lower-case base32, so mail servers that change an address's case don't break it. The key is derived from the application key: rotating APP_KEY retires the old reply addresses.
  • A UUID takes 47 characters, which leaves room for a prefix within the 64 characters an address allows before the @. Text values take more; address and address_for fail with InboundError::AddressTooLong past 64.
  • sign(value) / verify(token) work with text, sign_uuid(id) / verify_uuid(token) with UUIDs. A forged or damaged token is InboundError::InvalidToken.
  • tokens.scoped("announcements") gives tokens under a key of their own: a token signed for tickets doesn't verify as an announcement's, even for the same value.
  • Tokens don't expire unless mail.inbound.reply_token_ttl_days is set; an older token is InboundError::ExpiredToken.

Raw messages on a disk

Set mail.inbound.disk to keep every message's raw bytes as an .eml file, for audits, for an analyzer that runs later, or to show a message exactly as it arrived:

config/app.toml

The file is written before the mailbox runs, so a message whose handler fails is still kept. Its name is the message's SHA-256, so a message the provider delivers twice is stored once. The location is mail.stored in the handler, received.stored in the answer and stored in the event; Inbound::read(&stored) reads it back as an InboundMail. Use any driver: a local folder in development, an S3 disk in production.

Size limits

mail.inbound.max_size (default 25 MiB) is the largest raw message accepted. A larger one is refused before it is parsed: InboundError::TooLarge, a 413 with code mail_too_large. The source routes accept request bodies up to twice the limit, since providers that post JSON or forms encode the message in base64.

Sources

Each source is a public route under /api/mail/inbound, checked with its own secret and mounted only when that secret is set. Put the secrets in .env: APP_MAIL__INBOUND__POSTMARK__TOKEN=....

config/app.toml
RouteSourceChecked with
POST /api/mail/inbound/ses?token=..An SNS topic that SES receipt rules notifyThe token, and the topic when topic_arns lists any.
POST /api/mail/inbound/mailgun, /api/mail/inbound/mailgun/mimeA Mailgun route's forward()The HMAC-SHA256 of timestamp and token, at most 15 minutes old.
POST /api/mail/inbound/postmarkA Postmark inbound streamThe token as the basic-auth password, or a token query parameter.
POST /api/mail/inbound/rawRaw MIME from your own server or scriptAuthorization: Bearer <token>, or a token query parameter.

Each answers 200 with what became of the message:

JSON

A secret that doesn't match is a 401 with code invalid_signature, a payload the provider doesn't send a 400 invalid_inbound_mail, a message over the limit a 413 mail_too_large, and a failing mailbox a 500.

A provider posts every message from a handful of addresses, so its requests count against the per-IP rate limit (http.per_ip_rate_limit, 600 requests a minute by default). Over it, the route answers 429 and the provider delivers the message again later; raise the limit if your volume needs more.

Amazon SES

Verify your receiving domain in SES and point its MX record at SES, then create a receipt rule for the addresses you receive (in.acme.test, or single addresses). Give the rule one of two actions:

  • An SNS action publishes the whole message in the notification. It is the simplest, for messages up to 150 KB.
  • An S3 action with an SNS topic stores the message in a bucket and notifies the topic with its key. It works for any size. Configure an S3 disk on that bucket and name it in mail.inbound.ses.disk: the message is read from the disk, and since the bucket already keeps it, stored points at the object instead of a second copy.

Subscribe https://api.acme.test/api/mail/inbound/ses?token=... to the topic over HTTPS, with raw message delivery off. The route confirms the subscription by fetching its SubscribeURL (only https://sns.<region>.amazonaws.com; with confirm_subscriptions = false it logs the URL for you to open). The envelope recipients are the receipt rule's recipients, and the five verdicts come from the rule.

The route trusts the secret token in its URL and the topic list; it doesn't check the SNS message signature. To check that too, receive the topic with a verified webhook of your own and hand the notification to Inbound::receive_ses:

src/modules/mail/ses_inbound.rs

Mailgun

Create a route whose action forwards to https://api.acme.test/api/mail/inbound/mailgun/mime:

Text

A URL ending in mime makes Mailgun post the whole message as body-mime, which reaches your mailbox byte for byte. /api/mail/inbound/mailgun accepts Mailgun's parsed fields as well: the message is rebuilt from message-headers, the bodies and the attachment files (inline ones by content-id-map). The envelope recipient is the form's recipient; the spam flag, SPF and DKIM results become verdicts.

Postmark

Set the inbound stream's webhook URL to https://postmark:[email protected]/api/mail/inbound/postmark and turn on Include raw email content in JSON payload: the message then arrives as it was sent. Without it, the message is rebuilt from Postmark's fields (FromFull, ToFull, CcFull, the headers, TextBody, HtmlBody and the attachments), which keeps everything but the original MIME structure. The envelope recipient is OriginalRecipient; Postmark's spam status and SPF result become verdicts.

Raw MIME

Any mail server can hand a message to the raw route. With Postfix, a pipe transport for your inbound domain posts each message:

/usr/local/bin/navree-inbound

X-Envelope-From and X-Envelope-To (comma-separated) give the envelope; without them, mailboxes match the message's own headers. When the route doesn't answer 2xx, curl --fail fails and the script exits with 75 (EX_TEMPFAIL), so Postfix keeps the message and tries again.

Your own sources

Inbound::receive is the pipeline every route above calls. Hand it an Incoming from anywhere: a queue consumer, an IMAP poller, a test.

Rust

Incoming holds the raw message, the source's name, the provider id, the envelope, the verdicts and, when the raw message is already kept somewhere, stored. Incoming::from_postmark(&json) reads a Postmark payload; Inbound::receive_ses(&notification) reads an SES notification.

Microsoft 365 and Gmail mailboxes

Featureintegrations Some mail never passes through your own domain: the messages in a customer's own mailboxes, such as a shared security@ mailbox in their Microsoft 365 tenant or a support address in their Google Workspace. With the integrations feature on as well, MailboxWatches watches those mailboxes through the customer's integration connection and hands each new message to the same pipeline (source is graph or gmail).

The connection needs read access to mail: Graph's Mail.Read application permission for a Microsoft 365 tenant, or the https://www.googleapis.com/auth/gmail.readonly scope for Google Workspace (a service account with domain-wide delegation reads every mailbox of its domain). Give mail reading a scope set of its own, so a tenant can grant it apart from everything else.

config/app.toml

Start watching from a settings route, once the customer has connected:

src/modules/security/mailboxes.rs
  • Microsoft 365. watch creates a Graph subscription on the mailbox's inbox for new messages, with your graph_notification_url and a clientState derived from the application key. Graph checks the URL first: POST /api/mail/inbound/graph?validationToken=... echoes the token. Each notification then names a message, which is read as MIME from /users/{mailbox}/messages/{id}/$value. A notification whose clientState doesn't match is ignored. Lifecycle notifications are handled too: reauthorizationRequired renews the subscription, missed catches up, subscriptionRemoved marks the watch failing.
  • Gmail. Create the Pub/Sub topic, grant [email protected] permission to publish to it, and add a push subscription to https://api.acme.test/api/mail/inbound/gmail?token=.... watch calls users.watch on the mailbox's INBOX and keeps the history id it returns. Each push reads the mailbox's history since that cursor, fetches every added message in format=raw, and moves the cursor. When Gmail no longer knows the cursor, the watch restarts from the push's history id.

Watches are rows of integration_mailbox_watches (the model MailboxWatch): the connection, the provider, the mailbox, the Graph subscription_id or the Gmail cursor, the status (active or failing) with last_error, expires_at and last_received_at. The migration is part of the integrations set.

MethodWhat it does
watch(connection, mailbox)Starts watching, or renews the watch the mailbox already has.
renew(id), renew_due()Extends the subscription or the Gmail watch; renew_due renews those expiring within renew_before_hours, and failing ones.
sync(id)Reads what arrived since the cursor (Gmail) or the last message (Microsoft 365, a day at most), after an outage.
unwatch(id)Deletes the subscription or stops the Gmail watch, then the row.
all(), of_connection(id), find(id)Read the watches.
graph_notifications(&json), gmail_push(&json)What the routes run, for your own verified routes.

Graph subscriptions on mail last a few days and Gmail watches a week, so they must be renewed. integrations:mailboxes --renew runs every hour on the scheduler. integrations:mailboxes lists the watches, and integrations:mailboxes --sync <id> catches one up.

The Gmail route trusts the token in its URL. To check Pub/Sub's OIDC token as well, receive the push with a GooglePubSub verified webhook and call gmail_push yourself.

The event

Once a message is handled, rejected or found no mailbox, InboundMailReceived (mail.inbound.received) is emitted with EventsModule in the app. It describes the message without carrying it: id, source, provider_id, message_id, from, recipients, subject, size, attachments (a count), fingerprint, status, mailbox, address, stored and received_at. A queued listener reads the message back from the disk:

src/modules/archive/listeners.rs

Register it with m.listen::<InboundMailReceived, IndexInboundMail>(). A message whose handler fails is not announced; it is announced when the provider's retry succeeds.

Testing

Incoming::compose() builds a message field by field, and MailFake::receive plays the provider: it runs the message through the app's pipeline, as a source route would.

tests/tickets.rs
  • Incoming::compose() has from, to, cc, reply_to, envelope_to (a recipient the headers don't show), subject, text, html, message_id, in_reply_to, reference, date, header, attach, inline, forward (a whole message as an attachment), provider_id, verdicts and source; build() gives the Incoming and to_bytes() the MIME. Non-ASCII names and subjects are encoded as the real thing would be.
  • Real messages work as well: save a provider's .eml under tests/fixtures/ and receive Incoming::new("raw", std::fs::read(path)?).
  • fake.received() lists what became of every message the fake delivered; Inbound::mailbox_for(&mail) tells which mailbox would take a message without running it.
  • The source routes take the same payloads in tests as in production: post a fixture with the test client and the configured secret.
  • For watched mailboxes, Integrations::fake() stubs Graph and Gmail: fake.api().on(Method::GET, "https://graph.microsoft.com/v1.0/users/*/messages/*/$value", FakeResponse::new(200).body(mime)).

Errors

InboundError names what went wrong without quoting message bodies or secrets:

VariantWhenIn a route
TooLargeThe message is over mail.inbound.max_size.413 mail_too_large
UnparseableThe bytes hold no message headers.400 invalid_inbound_mail
MalformedA provider's payload isn't what that provider sends.400 invalid_inbound_mail
InvalidToken, ExpiredTokenA reply token is forged, damaged or too old.400 invalid_reply_token
AddressTooLongA reply address would pass 64 characters before the @.500
MissingKeyReply tokens need an application key.500
RejectedA mailbox rejected the message (ctx.reject).(not an error for the source)
InvalidPatternA mailbox pattern doesn't parse.boot error
ConfigA disk isn't configured, or an S3 action arrives without mail.inbound.ses.disk.500
StorageWriting or reading the raw message failed.500
HandlerA mailbox's handler failed; the provider retries.500

Watched mailboxes add IntegrationsError::MailboxWatchNotFound (404 mailbox_watch_not_found), and wrap pipeline errors in IntegrationsError::Inbound.

Reference

KeyDefaultMeaning
mail.inbound.max_size26214400The largest message accepted, in bytes.
mail.inbound.disknoneThe disk that keeps raw messages.
mail.inbound.pathinboundThe folder on that disk.
mail.inbound.reply_token_ttl_days0Reply tokens older than this are refused; 0 never expires them.
mail.inbound.ses.tokennoneThe SES route's secret; the route is mounted when set.
mail.inbound.ses.topic_arns[]The SNS topics accepted (all when empty).
mail.inbound.ses.disknoneThe S3 disk of the receipt rule's bucket.
mail.inbound.ses.confirm_subscriptionstrueFetch SNS SubscribeURLs.
mail.inbound.mailgun.signing_keymail.webhooks.mailgun.signing_keyMailgun's HTTP webhook signing key.
mail.inbound.postmark.tokennoneThe Postmark route's basic-auth password.
mail.inbound.raw.tokennoneThe raw route's bearer token.
integrations.mailboxes.graph_notification_urlnoneWhere Graph sends notifications; mounts /api/mail/inbound/graph.
integrations.mailboxes.gmail_topicnoneThe Pub/Sub topic of Gmail watches.
integrations.mailboxes.gmail_tokennoneThe Gmail push route's secret; mounts /api/mail/inbound/gmail.
integrations.mailboxes.renew_before_hours24Renew watches expiring sooner than this.
CommandWhat it does
integrations:mailboxes [--renew] [--sync <id>]Lists watched mailboxes, renews the due ones (hourly on the scheduler), or catches one up.