Burnbox — We Will Forget It for You, Wholesale

Delete is a lie. I watched it get proven in a courtroom.

In May, mid OpenAI-versus-Musk, a stretch of private executive chats got read into the record as discovery. "Private" turned out to mean on someone else's server, retained, subpoenable. Gone was never gone.

We give a lot to AI, and we never really stop to ask what we are giving up. Privacy apparently.

And here's why the delete button was never going to save you. When you talk to a normal AI, that conversation isn't in one place. It's on your device. It's in transit across the network. It's in the provider's logs. It's in the model's context, and maybe — later, quietly — in its next training run. Your conversation becomes part of tomorrows responses. One sentence you typed now lives in five systems you don't own and can't inspect, each one its own leak surface, its own subpoena target, its own breach waiting to happen. Same story for every message you send, every document you upload, every meeting you record. "Delete on my screen" does nothing about the other four copies. You cannot delete what you were never holding. Your data is the product. Your conversations are corporate property now. This is a scary thought when you consider the private things we throw into LLMs... medical conditions, legal advice, our deepest darkest secrets. We have all put things into ChatGPT we know we shouldn't have. Company documents. A file with our banking information, passwords, medical records... but it's ok... I can delete it right?

Deletion is a feeling we sell people. Theatre with a progress bar. Delete only means you can't see it anymore. OpenAI can. Anthropic can. The company hosting the model can. The courtroom that subpoenas the chat can. If it makes its way into training data... everyone can.

And this is not a niche gripe for the tinfoil crowd. Think about who actually needs to be sure. Someone trying to understand what their lawyer just told them, without bleeding privilege into a model. A teacher working out how to support a struggling kid, writing things about that child you do not want sitting on anyone else's server. An executive talking strategy that becomes discovery the instant it's retained. A doctor or a psychologist taking notes on a patient and thinking out loud about their care. A journalist using AI to work a story where the facts could get them detained or worse at a border crossing. Every one of them needs a privacy floor that a frontier model on somebody else's GPU structurally cannot give them. Their only honest options today are "don't use AI" or "use it and hope." Hope isn't a plan.

So I built Burnbox to kill the theatre. Seven weeks, 554 commits, late May to mid-July. The tagline is "the AI that forgets on command," and the whole project is the work of turning that sentence into something you don't have to take on faith.

Local-first desktop app. A Rust core across five crates, a React front-end, shipped on Tauri 2. Your conversations live in an encrypted vault on your own machine. The model that answers you runs inside attested hardware that physically can't keep what it sees. And when you burn something, you get back a signed receipt — a cryptographic artefact that says this exact thing was destroyed, here's the proof, verify it yourself.

Same principle I built Resumatron9000 on, pointed at a different enemy: trust is an architecture, not a disclaimer. Deletion has to be a ceremony that produces evidence, not a checkbox that produces a spinner.

The spine: five crates and a wrapped-key vault

The whole shape of the system fell out of one refusal. No server can see your plaintext, so there's no plaintext for anyone to leak. Everything else hangs off that.

The workspace is a Rust monorepo — burnbox-crypto (primitives), burnbox-agent (the LLM/agent loop), burnbox-mls (group-messaging key schedule), burnbox-sync (the zero-knowledge relay), burnbox-proxy (the enclave co-signer) — plus the Tauri app (~42k lines of Rust) and a React front-end (~24k lines of TypeScript). Release builds are lto = true, codegen-units = 1, panic = "abort", strip = true. A privacy binary you hand to strangers should be as small and as boring as you can make it.

The crypto is deliberately unexciting, which is the compliment. ~670 lines wrapping the correct, unsurprising choices — XChaCha20-Poly1305 for AEAD, Argon2 to derive the master key from your passphrase, X25519 for key agreement, Ed25519 for signatures, HKDF for derivation. Storage is SQLCipher. Nobody gets clever in a crypto core. Clever is how you get owned.

The load-bearing idea is the key hierarchy. Your passphrase derives a master key that never leaves the session, and that master key does exactly one job: it wraps per-object data-encryption keys. Every conversation, document, project and thread gets its own DEK.

That indirection is the whole trick. To burn something I don't chase its bytes across tables and caches and pray I got them all. I destroy its DEK. The ciphertext instantly becomes noise. The receipt is a signature over the hash of what I destroyed.

Build the exit first

The first seven commits are the entire thesis in miniature, landed on day one. Scaffold. Vault unlock. Encrypted conversations. SQLCipher persistence with wrapped DEKs. And then, before a single feature worth using: session 4, the burn ceremony. Signed receipts and a /verify path. Destroy a thing, sign a receipt, verify it with no server in the loop.

I wrote proof-of-deletion before I wrote the things worth deleting. On purpose.

If the ceremony had come last, it would've been retrofitted around whatever the storage layer happened to do — and retrofitting forgetting is exactly how you end up with shadows in a cache. Building it first meant every later feature inherited the same non-negotiable exit: it has a DEK, so it can be provably burned. Session 9 added auto-burn on a timer. Forgetting on a clock, not just on command.

Not one delete button — an arsenal

"Burn" isn't a button. It's a whole vocabulary of ways to forget, because different people are afraid of different things.

Torch a single turn and keep the rest of the chat. Kill a whole conversation, or an entire project and its distilled memory, in one ceremony. Set a lifetime and let a thing forget itself on a clock. Type a panic phrase and watch everything go, right now.

Then the ones built for people who are actually at risk.

A duress passphrase — a decoy. Enter it at unlock and Burnbox secure-wipes your real vault and opens a separate, plausible, boring one in its place. A coercer standing over your shoulder forcing you to log in sees a vault. Just not yours. Device-only, no network round-trip, the burn already done before they've finished reading the screen.

A dead-man switch — miss a check-in and it burns on its own. Protection for the exact moment you can't reach a keyboard: detained, hospitalised, worse. A geofence — leave your safe zone and a burn can trip. Ghost chats — RAM-only conversations never written to disk in the first place, so there's nothing to burn because nothing ever existed. And burn-for-both — a signed, replay-guarded remote burn, so "delete for everyone" is a cryptographic event with a receipt, not a request you hope the other end honours.

Every one of them, bar the emergency duress wipe, hands you back a signed receipt. Forgetting, itemised and provable.

And that list of people from up top isn't decoration. The journalist gets an insurance file — a sealed document the relay releases only if you stop checking in, the cryptographic version of "if something happens to me, publish this" — plus a whistleblower packager that scrubs, bundles and signs a set of files into one self-verifying package, and proof-of-existence to notarise a file's hash so you can prove this exact thing existed at this time without ever revealing the thing. Redact-before-send strips EXIF, GPS and Office-doc metadata before a byte leaves. The tools match the fear.

Proving the model can't keep what it never stored

A fortress on your device is worthless the second your words hit someone else's GPU.

So inference runs on attested open models inside Intel TDX enclaves on Phala, and every response comes back with a per-response attestation signature naming the model that produced it.

But an attestation you can't check yourself is just a green tick with extra steps. So the second signature on every receipt comes from burnbox-proxy, a ~540-line Axum service in its own Phala CVM. The client does the interesting part. On first contact it fetches the attestation, pins the Ed25519 key, and re-verifies the TDX quote locally — parses the report data, checks the quote actually binds the signing key, walks the quote chain itself. It never trusts the proxy's own attested: true. It proves it. Mid-session key swap? Rejected. Proxy unreachable? The burn still completes with a device-only receipt — the guarantee degrades, it doesn't fail shut.

You end up with two Ed25519 signatures over the same canonical bytes, checked by one shared verifier that both the desktop and the public web verifier run — so nobody's ever checking a subtly different thing than anyone else.

A client that re-checks the quote is doing security. A client that reads attested: true off the wire is doing theatre. I built the first one.

The audit log is not allowed to become the leak

If your pitch is "we forget," the thing people most need to watch is what leaves. So there's a per-conversation egress ledger: a live record of every outbound byte — which model call, which fetch, which tool, how much — with plaintext never in the ledger itself. State-changing tools sit behind consent gates and per-task scoping. Revoke a consent after the fact and it's gone.

The discipline that demands showed up in one tiny bug I refused to ship.

Burnbox has a geofence — anti-coercion, trips if the device leaves a safe zone — and defining a zone needed address search and a one-tile map preview, a little Rust geo_lookup hitting Nominatim and a tile server. The trap: a lazy ledger entry reading fetched map tile (x, y, z) would have written the exact location the feature exists to protect straight into the audit log. The privacy control leaking the private thing into the privacy record.

So the summaries carry lengths and zoom levels. Never the query. Never the coordinates.

That's the entire philosophy in one refused commit. Privacy isn't a feature you bolt on. It's an invariant you won't break even in the plumbing. If your invariant doesn't survive contact with your own telemetry, you don't have an invariant. You have a slogan.

Memory that's scoped and destructible

A useful assistant remembers things. Memory is the natural enemy of forgetting. The naïve version — one global memory store — is a leak with a UI.

This is the exact inverse of Rekall, and I built both at once, which was a strange and useful place to stand. Rekall makes memory persist and compound. Burnbox makes it scoped and destructible. Here the answer is per-project memory sealed under its own DEK: what the assistant learns in one project is encrypted with a key only that project holds, injected at a single load chokepoint so it can't bleed into a chat where it doesn't belong, distilled on the write path, and destroyable wholesale by a burn_project ceremony that — obviously — emits a receipt.

The store is at schema v21, reached through 21 forward migrations, and that number is its own honest little story. Every one applied cleanly on a fresh database in CI before it shipped, because the one thing you can never do to a privacy vault is corrupt it on upgrade. The rule I gave myself and kept: checkpoint before an irreversible migration, never after.

The agent grows a spine

Mid-June the app stopped being a chat box and became a workbench. Keep-mounted per-conversation panes, so switching surfaces doesn't tear down a running stream — that one came out of an early bug where stream events bled between conversations mid-switch, which taught me to make session identity explicit instead of hoping. Then the loop proper: a unified controller with hooks, triggers, budget caps and task metrics; an ask_user tool that slides a real question-card into the conversation and blocks on your answer; live steering mid-run.

One piece of design I'm fond of: AgentProvider is a supertrait of LlmProvider, so the plain "stream me tokens" contract and the richer "run tools, spawn sub-agents, compact context" contract compose instead of forking. That let all the later reliability work — cross-provider failover, parallel sub-agent fan-out, context compaction for long runs — slot in without rewriting the chat path. On top of it sits a catalog of safe-rendered UI primitives and a Cockpit of pinned, refreshable, burnable widgets, each with their own decoupled keys and sealed ledgers.

Messaging, and a spike I'll own out loud

The messaging epic is where the cryptography got its hardest workout. Disappearing 1:1 messages, then it hardened fast: signed group messages for author authenticity, group key epochs so removing a member actually rotates the key (forward secrecy, not a UI lie), then recipient-side forward secrecy via one-time prekeys. Identity is trust-on-first-use with teeth — the address-to-signing-key binding pinned at add-time, a surfaced warning on key change, a re-verify ceremony. The honest version of safety numbers, where a silent key swap is a visible event, not a shrug.

Now the spike, because the whole brand of this thing is honesty and I'm not about to fake it in the write-up. I stood up an isolated burnbox-mls crate as an OpenMLS (RFC 9420) proof. The full MLS tree is the right long-term answer for large groups. But the shipped path today is the lighter signed-sealed-send scheme, and that MLS crate sits at ~120 lines as a proven seam, not a load-bearing wall. Saying that plainly is cheaper than implying the whole RFC is wired in and getting caught. The rest of the epic was product discipline: a unified inbox, one background poller instead of a Refresh button, content-free notifications, and burn-for-both — a signed, replay-guarded remote burn, so "delete for everyone" is a cryptographic event with a receipt instead of a request you hope the other client honours.

Voice, and the boundary audio never crosses

The clearest "invariant, not feature" in the whole build, and the most recent chapter. Burnbox does live dictation and on-device meeting transcription under one hard rule: audio never crosses the Tauri boundary. Never the Rust core. Never disk. Never the network. The mic is captured and transcribed entirely in the renderer via WASM, and only the text ever reaches the app.

That constraint made it genuinely hard, which is exactly why it was worth doing.

My first streaming cut flushed fixed windows and appended them — and cut words in half at every boundary, reading like a bad fax. The fix was to stop appending and re-transcribe the whole utterance-so-far every ~950ms, replacing the span in place so words firm up as context arrives, guarded so two ticks never collide on the single ASR pipeline.

But that only worked on the right model, and I measured instead of guessing. Behind a dev-only route, I benchmarked Moonshine against Whisper: Moonshine transcribes a one-second clip in ~78ms; Whisper pads every clip to 30 seconds and lands flat at ~3s. One keeps pace with speech. The other structurally can't, no matter how you tune it. Second gate: drive it on an iPhone simulator and prove it loads and infers without blowing the memory ceiling. WKWebView doesn't expose performance.memory, so the test was brutally simple — does the app survive? Both green, I rewrote the pipeline on the fast model and deleted the spike.

And the payoff is the part those people in the opening actually need. The transcript is speaker-diarized and durable — Alice:… Bob:… — and because it never leaves the machine, the sensitive thing you do next stays on the machine too. Turn a raw transcript into a SOAP note if you're a doctor, a therapy note if you're a psychologist, a write-up if you're a journalist, minutes and action items if you're the one in the meeting that becomes discovery. Structured output, template-driven, on-device, never round-tripped through someone else's server.

The privacy requirement is what made the problem hard. The on-device answer turned out to be the better product and the stronger guarantee. That happens more than you'd think.

The honest reality check

The heart of the system — wrapped-key vault, signed-burn ceremony, two-signature receipts with local TDX re-verification, egress ledger, per-project sealed memory, forward-secret messaging, on-device voice — is built, and covered by ~460 Rust tests and ~370 TypeScript/e2e tests, including the boundary tests that would catch a transcript leaking across the IPC line or a burn chain breaking.

What's left is the unglamorous last mile that only credentials unlock. Standing burnbox-proxy up in a production Phala enclave — the CVM compose and the deploy runbook exist, it needs the account token. Code-signing and notarising the desktop build so it installs without a Gatekeeper scare — the CI's wired for it, Apple secrets sitting as no-ops until they're set, it needs a Developer ID. Those aren't design problems. They're a deploy and a certificate away. I'd rather say that than pretend the ribbon's already cut.

And I'll own the standing cost, because it's the honest part. A privacy product is a one-way ratchet. Every convenience — sync, memory, widgets, a maps preview — had to be re-argued against the invariant instead of waved through. That's slower. It's also the whole point.

What I'd pull out of all this

1. Deletion is a claim, and claims need evidence. Ship the proof of forgetting, not a spinner. A burn without a receipt is a rumour.
2. Build the exit first. Write the burn ceremony on day one and every later feature inherits a provable way to die. Retrofit forgetting and you get shadows in a cache.
3. Wrapped keys make forgetting cheap and total. Give every object its own DEK and deletion becomes "destroy one key," not "chase bytes across tables."
4. Don't trust an attestation you can't re-verify yourself. Re-checking the TDX quote locally is security. Reading attested: true off the wire is theatre.
5. The audit log must not become the leak. If your invariant doesn't survive contact with your own telemetry, it was never an invariant.
6. Constraints make the engineering. "Audio never leaves the renderer" is why dictation was hard — and on-device turned out stronger, not a compromise.
7. Forgetting and remembering can coexist — if memory is scoped and destructible. Per-project keys turn "it remembers too much" into "it remembers exactly this, and I can burn it."

I built Burnbox because I was done with privacy being a promise. The whole thing replaces trust me with check it yourself — and takes the most human thing an AI can do, forget on command, and makes it something you can hold in your hand and verify.

If you want a guarantee that's cryptographic instead of aspirational, that's the kind of thing I build.

And unlike the big guys, there are some things Burnbox just can't do. Like leak your private conversations.