dangerouslyskip
← Back to writing
Essay·Jun 20, 2026·8 min

Human in the Loop Is a Trap

Turn-taking is the rendering. Memory is the runtime. We solved multiplayer for the machines and left the humans alone in a corner.

We spent two years teaching machines to work together. We used the same two years to make humans work more alone than ever.

That's not a paradox. It's a screwup, and it's hiding in plain sight.

Go look at any modern agentic system. You'll find a small miracle of coordination — an orchestrator delegating to a swarm of agents, each taking a slice, negotiating ambiguity, tolerating each other's latency, picking up where the last one left off, reconciling the mess into something coherent. The machines are multiplayer. We cracked it. The two-hundred-thousand-year-old problem of getting independent actors to collaborate without trampling each other — we shipped a workable version for software in about eighteen months.

Then we sat exactly one lonely human in front of the whole apparatus and handed them a chat box.

That's the asymmetry nobody's naming, and it should be keeping people up at night. The agents got multiplayer. The humans stayed resolutely single-player. From chat to the IDE, every primitive we built is single-player by default — one person, one session, one context window, copy-pasting fragments into Slack so anyone else can see what they're doing. We orchestrated the robots into a team and left the humans as fragmented individuals, each holding their own little candle, lighting their own little patch of dark.

We painted ourselves into a corner. We did it while congratulating ourselves on the orchestration.

Webb's near-miss: the turn is a red herring

The most useful thing I've read on this is Matt Webb's piece on multiplayer turn-taking — useful precisely because it's a brilliant near-miss. Webb takes a chatroom full of humans and bots and solves who speaks next: a clever "enthusiasm" score, grounded in the real social-science literature on conversational turn-taking. Genuinely good engineering. And he's dead right that even one human with one AI is already multiplayer — two actors sharing an environment.

Here's where I part ways with him, and the parting is the whole point.

Webb did superb functional work on top of a structural mistake. He accepted the turn as the primitive — synchronous, sequential, who-speaks-next — and did clever things inside it. That's optimising the square peg. Turn-taking was never the problem. It's a symptom of accepting the wrong substrate, a red herring that feels like the hard problem because it's fiddly, while the actual problem sits one layer down, untouched.

Turn-taking is real — at the speech layer. The live, co-present, 200-millisecond-gap, gaze-and-prosody layer his own citations describe. That layer runs on turns. But that's not where the multiplayer-AI fight is. The fight is at the collaboration layer — teams, orgs, async, work that unfolds over hours and days — and there, humans do not reason turn-by-turn. We never have. Reason about a group project and you don't replay the conversation message by message. You query a model of it: who knows what, what's been decided, what's still open, who's good at which thing. We reason from memory, not from a transcript.

And the kill shot is hiding in Webb's own code. To decide who speaks next, his system has to constantly reconstruct a little state model of the room — who was addressed, who spoke recently, whether there's a sub-conversation in flight. Memory was doing the real work the whole time. He just kept it subordinate to the turn instead of flipping the hierarchy.

Which is the line this whole essay turns on:

Turn-taking is the rendering. Memory is the runtime.

The turn is the UI. Memory is the database. Everyone keeps lovingly optimising the UI. (Same disease, different organ as Chat Can’t Be the Answer — chat is intrinsically turn-based, and turns are the wrong primitive for work.)

Get that, and the whole prescription falls out of it. Swap the primitive from turns to memory and watch what stops being hard.

What memory-as-primitive unlocks

Async stops being a feature and becomes the default. In a turn world you have to be present to take your turn — the synchronous bottleneck is baked straight into the primitive. In a memory world, every participant reads and writes shared state whenever, and an orchestrator reconciles. You don't add async. You get it free the moment you stop building on turns.

Which means you can finally treat the human as just another agent in the stack.

I know how that sounds. The careless version of that claim is dehumanising and deserves the dunking it'd get, so let me be exact: this is the opposite of "humans are cogs." Right now the human is the cog — the synchronous chokepoint the whole pipeline blocks on, waiting for them to respond, approve, weigh in. Treat the human as an agent the orchestrator schedules around and the system absorbs their latency the way it already absorbs a slow tool-using agent's: routes around them, proceeds with what it can, queues what genuinely needs them, folds their judgment in when it lands. The human stops being the thing everything stalls on and finally gets the same first-class async treatment we engineered for the machines and then selfishly kept from ourselves.

It's not a demotion to peer. It's a promotion out of bottleneck.

And that's what springs the trap. "Human in the loop," as practised, is a turn-based idea wearing a compliance lanyard. The human is a mandatory turn the pipeline blocks on — and usually a context-poor human grading the output of context-rich agents. Sit with how backwards that is. The agent read the whole codebase, the whole ticket history, the whole blast radius. The human glances at a diff for eight seconds and clicks approve.

That's not oversight. That's theatre. Latency and false assurance, dressed up as control. (The exact cousin of the approve-approve-approve button I went after in Kenny Loggins Mode — a thing that made us feel safe while doing nothing.)

The human's edge was never grading. It's deep domain judgment on the few calls that actually need it — the ones only someone with the scars can make. So the move isn't "human in the loop" everywhere. It's human in the right loop: route the decisions where human expertise genuinely dominates to the human, let the agents own the rest.

Now the objection, before someone hands it to me: in regulated, high-stakes, liability-bearing work you can't just delete the human gate. Accountability isn't optional, and I'm not pretending it is. But that's an argument against blanket, low-context grading — not against oversight. Keep humans for real judgment and real accountability. Stop burning them as rubber stamps. And the machinery to do this safely is buildable, because I built it: Echo earns the right to auto-approve low-risk work so humans only see what deserves a human, and Governor makes every team's policy travel with the work as code, asserted in CI, instead of as a meeting the pipeline waits on. The answer to "but who's accountable?" isn't a mandatory human turn. It's policy-as-code plus earned, inspectable autonomy.

The guardrails replace the tollbooth.

From hunger games to a three-way powerhouse

Here's why it matters past the architecture diagram. The enterprise conversation about AI offers you two games, and both are zero-sum. Human-versus-human — the old siloed rivalry, who ships more, who gets the promotion. And the one everyone's actually scared of: human-versus-agent, the replacement hunger games, who gets automated away.

Both are losing games. Both are what you get when the primitive is the lonely turn and the only question is who takes it.

Change the substrate to shared memory and a third game appears — the only positive-sum one on the board. Human-human-agent, collaborating as a three-way knowledge powerhouse: human domain judgment, human-to-human trust and context, agent breadth and tirelessness, all reading and writing the same shared state. And it isn't wishful thinking. It's the one place the data's already in — a 2025 Harvard Business School study of 776 professionals found human-plus-AI teams beat both solo AI and humans alone.

The three-way beats both binaries. We have the receipt. We're still building for the binaries.

The primitives, briefly, because I'd rather show than sell

Not hand-waving, and I won't turn it into a brochure, so fast: if memory is the runtime, you need real shared memory — which is why I built Rekall (personal, hive and agent memory, with provenance, so institutional knowledge stops walking out the door with every resignation) and Skynet (shared sight across the whole codebase, so context is ambient instead of copy-pasted). You need to shift the org's input left, async, into the moment of work instead of a downstream review — that's Governor. And you need an orchestration layer that treats a human as a schedulable peer, not a blocking gate. I've built versions of each, which is mostly how I know they're buildable and not just bloggable.

Shared memory isn't the novel bit — vendors already sell it. The novel bit is that memory isn't a feature of multiplayer. It's the primitive. It's the thing that replaces the turn.

Let the humans back in the room

So here's the landing. We did something remarkable: we taught the machines to work together. Then we used the breakthrough to seat every human alone at a chat box, taking turns with a robot, blocking the whole pipeline every time they look away.

The frontier from here isn't a smarter agent. It's letting the humans back into the room — as peers in the same orchestrated, memory-backed system, not lonely operators stood outside it feeding it prompts one turn at a time.

Stop optimising the rendering. Build for the runtime. Do that, and the humans get to be a team again — this time with the machines in the room.

Grab a weapon from the cornucopia, kids. Welcome to the 101st Hunger Games.

ShareLinkedInX
Subscribe

New essays, build logs and experiments — in your inbox, occasionally. No spam, leave whenever.