Paper Trail: Every Mailbox, One Archive

An agent searches my mail. It has never seen a Gmail token, an Outlook API, or a login page, and it never will.

The rule

I wanted an LLM to answer questions about my email: when did that invoice arrive, what did the contract say, who wrote to me about this in 2019. The obvious move is to give the agent OAuth access to Gmail and Outlook and let it call the APIs.

I did not. The rule, written in the design doc before the first line of code: the LLM has no access to the Gmail or Outlook account. Not through OAuth, not through an API, not through the web interface. Not at runtime, and not during development either. Only through a tool I build myself.

So the tool, emailexporter, is split in two:

• sync and login talk to Google and Microsoft. A human runs them.
• Everything else - search, show, thread, att, contacts, events, stats - reads only the local archive and has a --json flag. That flag is the agent interface.

The agent can read everything I have downloaded. It cannot download, send, delete, or log in to anything.

N mailboxes, one archive

Paper does not arrive in one place, and neither does email. My mail is spread over several accounts, and the useful question is never “what is in this inbox” but “what do I have”. So the tool is multi-account from the start: one entry in the config per account, and every account lands in the same archive and the same index.

Phase one is two accounts: my Gmail, which goes back to around 2004 and weighs about 6.5 GB, and my personal Outlook. The rest of my addresses come later, each one more line in the config.

The same message often lives in two accounts: forwarded, CC’d, sent from one to the other. The index groups copies by Message-ID and shows each message once.

Raw first, everything else derived

sync   →  raw/            original .eml files, immutable, SOURCE OF TRUTH
parse  →  attachments/    extracted attachments        (derived)
       →  index/mail.db   SQLite + FTS5                (derived)

One plain .eml file per message, raw/<account>/<YYYY>/<MM>/<id>.eml. File exists = message downloaded, so an interrupted sync resumes for free. Writes go to a temporary file and an atomic rename, so a half-written file never counts. Spam and Trash are not downloaded.

Everything else can be rebuilt from raw/ without internet: emailexporter index --rebuild. If I change the parser, the search normalisation or the schema, I throw the index away and rebuild it. The originals are never touched.

Two providers, two sets of surprises

	Gmail	Outlook
API	Gmail API	Microsoft Graph
Scope	gmail.readonly	Mail.Read
Incremental	historyId	delta query per folder
Organisation	labels (many per message)	folder (one per message)

Read-only scopes on both. Two things worth knowing before you start. A Google OAuth app left in testing mode issues refresh tokens that expire after 7 days, so it has to be switched to In production (unverified, you accept the warning). And Graph returns MIME that Exchange reassembled: complete content, but not bit-identical to the original message.

Search that understands Serbian

The index is SQLite with FTS5. The text is normalised before it is indexed: lowercase, diacritics folded (đ to dj), Cyrillic transliterated to Latin. So рачун, račun and racun are one word. Every search term also works as a prefix, which covers grammatical cases: racun finds račun, računa, računu.

A second FTS5 index with the trigram tokenizer handles parts of words: invoice numbers, addresses, the middle of a reference code. Text extracted from PDF, DOCX and XLSX attachments goes into the same search. Attachments are stored once by SHA-256, however many messages carry them.

Real fuzzy search, the kind that forgives typos, is not there yet. If it is ever needed, the plan is Tantivy, not more SQLite.

Semantic search is not there either. I am considering a plain vector database next to the index, wired by hand, so that a question can find a mail that shares its meaning but not a single word. I am not sure how much it would add. The agent already rephrases a query and searches again when the first attempt misses, which is most of what embeddings would do here.

Newsletters are flagged as bulk from the List-Unsubscribe, List-Id and Precedence: bulk headers, and hidden by default.

The CLI is not for me

All of it sits behind one command line. The LLM built it, and the LLM is the only one who uses it. I do not know a single command. I have never typed one. I just ask, in Serbian, the way I would ask a person. Real requests:

• “ajde prodji sve emailove od knjigovodje - jel mi slala nesto sto nisam odradio?” (Go through every email from my bookkeeper - did she send me anything I never handled?)
• “ajde pogledaj sta sam sve poslao knjigovodji i dobio od nje - mozes napraviti mini grafikon poslati i primljenihj emailova” (Everything I sent to and got from my bookkeeper, as a small chart of sent and received.)
• “ajde vidi koje filtere za vazduh imam bro” (Which air filters do I have?)
• “ajde syncuj i proveri jel sve dobro poslato” (Sync and check that everything went out.)

Behind the bookkeeper questions, the agent ran things like:

emailexporter search --from <bookkeeper> --since 2026-10-04 --json --limit 20

then read the JSON, opened the thread, and answered. Which flags exist, how the query syntax works, what the output looks like: that is between the agent and the tool. The --json flag on every read command is there for the agent, not for me.

That is the real interface. Not a search box, not a CLI, but a conversation, with a tool underneath that the model knows better than I do.

In my opinion this works significantly better than any search built into an email client. It searches every account at once, one copy per message. It does not care whether the sender wrote in Cyrillic, Latin, or Latin without diacritics. It finds a word in any grammatical case, and it finds it inside the PDF attachment, not only in the mail body. And it can follow up: open the thread, read the attachment, compare two invoices, which no search box does.

Two sources, one agent

For documents, the family archive already has this: paperless, an API over everything scanned or forwarded, and the agent is the one who talks to it. Now the agent has a second source, for email, plus the contacts and calendars that come with the same accounts.

The point is to use them together. Paperless knows what a document says. The mail archive knows who sent it, when, and what was written around it. One question can go to both, and each fills the gaps of the other.

And the agent can move things across. An invoice or a contract found in the mail archive goes straight into paperless, through its API. Both sides already think in hashes: the archive stores every attachment by SHA-256, paperless detects duplicate files, so the same PDF does not end up in the archive twice.

Built for one person

This is a personal project for my personal email, and it is scoped that way on purpose. I know what it does not need:

• an enterprise-grade threat model;
• isolation against a malicious process already running under my user;
• perfect deduplication for every possible broken Message-ID;
• more infrastructure just because edge cases exist.

One machine, one user, a few accounts. Good enough for that is the target, not a compromise.

Security, now and next

Now: the archive lives on an encrypted LUKS volume, OAuth tokens live in the system keyring and never in a file, and the backup goes through restic, which encrypts before anything leaves the machine.

Next: the tool encrypts the archive itself, so a copied folder is useless without the key. Today the protection is the disk (LUKS); once the machine is unlocked, every file in the archive is plain text to every process running under my account.

The plan uses age, a small, modern file-encryption format with short keys and no configuration, called from Python through pyrage. It is asymmetric: the public key locks, the private key unlocks. Sync needs only the public key, so it can run unattended, even on another machine, and still cannot read what it downloads.

• Raw mail: every message compressed with zstd, then encrypted: <id>.eml.zst.age. Plain .eml files are gone; reading goes through the tool.
• Attachments: the same, and the file name becomes an HMAC instead of a plain SHA-256. A plain hash of a known file is enough to prove you have that file; an HMAC needs the key.
• Index: SQLCipher, an encrypted SQLite. FTS5 search works inside it unchanged.

The open question is where the private key lives. In the keyring, it protects against a stolen disk. Behind a password typed at serve and held only in RAM, it also keeps out every process running under my account, including LLM agents. That is not malware protection; a malicious process under my user has already won, and the list above says so. It is a boundary for the agents: they read the archive only while I have unlocked it. Lose the private key and the archive is gone, so a copy lives in the password manager.

Conclusion

Paperless for documents, the mail archive for email, contacts and calendars. Two local sources, one agent in front of both, and me asking questions in Serbian without knowing a single command. The LLM reads everything I have downloaded and files what it finds into paperless, and it has never logged in to a mailbox.

The agent gets my mail through a door I built, and I am already designing the lock for that door.

Now I only need to find the backup of the beotel.yu mailbox I used in the late 90s, from a Serbian ISP on a top-level domain that no longer exists (Yugoslavia’s .yu was switched off in 2010), and I will be invincible.

---

The Paper Trail series

1. Paper Trail - the map of the whole project
2. Paper Trail: Bootstrapping a Family Archive - the software bootstrap
3. Paper Trail: Phone, Flatbed, or a Real Document Scanner - the hardware door
4. Paper Trail: Push, Pull, and No App at All - how the paper actually gets in
5. Paper Trail: Every Mailbox, One Archive - this post, the second source: email, for the same agent