The numbers are in and they're ugly.
MIT's NANDA study: 95% of enterprise GenAI pilots produced zero measurable return — with a learning gap at the centre of it. Gartner, having earlier forecast trillions in agent-mediated spend, now predicts more than 40% of agentic AI projects get cancelled by the end of 2027: escalating costs, unclear business value, inadequate risk controls. Forrester finds three-quarters of enterprises chasing agentic AI and almost none scaling it.
Every one of those reports lands on cost, governance and integration.
All real. All downstream. All symptoms.
We sold the market a digital colleague and shipped it a contractor with anterograde amnesia. Then we act surprised the ROI doesn't land.
Which brings me to the title, and to Tommy.
The Who's rock opera is about a boy who is deaf, blind and mute — sealed off from the world after a childhood trauma — and who turns out to be a pinball champion. He plays by feel and vibration and instinct, and he beats everyone who has ever played. The crowd can't believe it. That's the line at the top of this post: astonishment that a kid with no senses is extraordinary at the one game in front of him.
That's your agent.
Genuinely brilliant at the machine it's standing at. Deaf and blind to the entire building it's standing in.
Three faculties
Think about the person an agent was pitched as replacing. Not a junior. The twenty-five-year veteran.
What makes them expensive isn't raw intelligence — the agent probably wins on that. It's three things they've accumulated that the agent has no mechanism to acquire.
They learn. Break something today, recognise it tomorrow, never do it that way again. And crucially, they tell the team, so nobody else does it either.
They remember. Not transcripts — decisions. Why the billing job runs at 3am. Which vendor you tried in 2019 and why it went badly. Who to ask. That's not recall, it's judgment with a history attached.
They perceive. They notice the deploy feels wrong before the alert fires. They know this service always wobbles when marketing runs a campaign. Their world isn't a text box; it's the whole workplace, all channels, all the time.
Learning. Memory. Perception. That's the gap. Not governance, not cost — those are what the gap costs you.
(Maria Sukhareva makes a version of this argument in The Real Promise of Agentic AI, and Karpathy's blunter framing is that agents are cognitively lacking and don't do continual learning. Both worth your time. I'm interested in what you build about it.)
All of these are buildable, and none of them require a frontier breakthrough. They just don't live in the model, which is why this reads as a map rather than a whinge. The opportunity is sitting there mostly untouched.
(There's a fourth I'd add to the list, and I'll get to it — it's the one costing you money right now.)
Learning is a multiplayer activity
Start with the one everyone's waiting on, because the waiting is the mistake.
The implicit industry plan is that continual learning arrives in the model. One day the weights update as you go, the agent learns your business, problem dissolves. Maybe. Not soon, and not per-company — nobody is fine-tuning a frontier model on your incident history every Tuesday, and if they were you'd have created a compliance problem that makes the original one look quaint.
The weights froze at training. They stay frozen in production. Your feedback changes nothing, no matter how many times you correct the thing.
So stop waiting. If the model can't learn, the architecture has to.
Picture architectural learning and you probably picture a single agent getting better at its job. Remembering your preferences. Learning your codebase. Improving its own prompts. Personal, private, single-player.
That's not learning. That's a smarter contractor.
Organisational learning is inherently multiplayer. The veteran's twenty-five years aren't valuable because they know things. They're valuable because they told someone, and the team stopped making that mistake, and eventually the way the org works quietly encodes a lesson somebody learned the hard way in 2009. One person learns. Everyone inherits. That's the actual mechanism, and it's the one we've comprehensively failed to build.
A correction that teaches one agent is a rounding error. A correction that becomes a rule every agent and every human inherits, automatically, is compounding infrastructure.
That's what Governor does. Vertical teams — cyber, cloud, data, legal-risk-compliance — assert their requirements as policy-as-code, injected at invocation and hard-asserted in CI. So when someone learns the expensive lesson, it doesn't get written up in a Confluence page nobody opens. It becomes a rule that travels with the work, pushes back on the generative function at the moment of work, and gates the merge when it's violated.
Learn once. Enforce everywhere. Forever.
That's the org-scale learning loop, available today, with technology that already exists. It just doesn't look like the thing people are waiting for, because it happens in the harness rather than the weights.
Memory: the difference between recall and judgment
The second faculty, and the one I've spent the most time building.
Most "AI memory" is a bigger notepad — it remembers what was said. Useful, nowhere near enough. The veteran's advantage was never recalling conversations. It's remembering what was decided, why, and how it turned out — and refusing to do the dumb thing twice.
That's the bet behind Rekall: memory as infrastructure, not a feature. Personal, hive and agent scopes, so what one person learns lands in the shared hive mind instead of dying in their skull. A decision graph where choices are first-class nodes (chose_over, based_on_evidence, superseded_by) and you can record the outcome, so the system later tells you you tried this, here's how it went. And a validation gate that won't let an agent act against something already confirmed.
Note the shape: same multiplayer principle as Governor, different substrate. Governor makes a lesson enforceable. Memory makes it retrievable, with its reasoning attached. You want both, because some lessons are rules and some are judgment — and treating judgment as a rule is how you end up with a compliance department nobody respects.
Twenty-five years of hard-won context, externalised and shared. Memory doing learning's job, because learning can't do its own.
Perception: the one almost nobody's building
The third faculty has the least competition around it, which is why it's where I've been spending my time.
An agent perceives only what reaches its context. Text, some images, maybe video. Its observable world is digital, thin, and entirely secondhand. It has no peripheral vision, no ambient awareness, and no idea the building is on fire unless someone types "the building is on fire" into the chat.
Meanwhile enterprises are drowning in signal. Alerts, tickets, logs, deploys, incidents, comms, market noise, customer complaints, the lot. Almost none of it reaches the agent. We built brains, gave them no senses, and wondered why they don't notice things the way colleagues do.
The obvious move — shovel everything into the context window — isn't perception. It's a firehose pointed at a straw. Perception is ingestion plus triage: knowing which of ten thousand signals matters right now, and which three are the same event wearing different hats.
Which is a solved problem. Just not in our field.
Security operations cracked this twenty years ago, because a SOC drowning in alerts was the original version of this problem and they had no choice. That's the prior art I'm building Beacon on — MISP's patterns, not its code: the sensor/module system, the correlation engine design, the taxonomy model, the sighting pattern.
A sensor, in Beacon's language, isn't the third-party tool. It's the adapter that translates one — the Jira sensor, the Slack sensor, the deploy-pipeline sensor. Sensors are the ears and eyes. Beacon is the brain.
The hard part is what happens after ingestion, and it's what separates this from a log aggregator. Signals about the same event share no identifiers. A support ticket about "slow checkout," a Splunk alert about database latency, and an engineer's Slack message about query optimisation are all the same thing happening — and no exact-match correlation on earth will join them. So correlation has to be deterministic and semantic: embed every normalised signal, cluster by meaning, run synthesis over the emerging clusters on a heartbeat. That's the system noticing the smell before the alarm.
Then the SOAR half — triage — because noticing isn't enough. Weighting lives in taxonomies: a signal-authority axis (CEO directive versus automated metric versus one person's complaint), business impact, strategic alignment. Composite score from the tags, with an adjustable formula, because "strategic alignment matters more than customer volume this quarter" is a real thing a human should be able to tell the system. An agent doesn't need ten thousand signals. It needs the three that matter, ranked, with provenance attached.
Underneath sits a temporal signal graph, because facts change — and a perception layer that can't represent "this was true in March and isn't now" is just a very expensive inbox.
Sensing, correlation, triage, orchestration. Security built that stack because their systems were drowning long before ours were. We should steal it wholesale instead of pretending perception is a context-window problem.
And perception is multiplayer too — one team's signal is another team's early warning. A shared sensory layer means the incident cyber noticed at 2am informs the deploy platform ships at 9.
The fourth faculty: nobody built the exam
Those three are the ones the discourse has settled on. Here's a fourth I'd add, and I think it might be the most immediately expensive of the lot.
An agent has no way to know what "done" means in your organisation.
Think about how these things were raised. Every agent grew up inside an evaluation environment where a grader existed — try, get scored, try again, until the environment says the work is acceptable. That's the entire shape of its upbringing. Then it graduates into your company, where nobody has built the exam.
So it finds a finish line it can recognise. You ask for a business outcome and you get a plan, a report, a chain of reasoning, a folder of artefacts, twelve status updates, and a request for approval. An industrious hour later, nothing about the company has changed. The agent didn't refuse the work. It found a different way to finish.
You are paying for activity and calling it work.
Nate has the two examples that make this land, in You Are Paying for Agent Activity and Calling It Work. OpenAI ran roughly 1,200 experimental agents against cybersecurity problems; a chunk of the targets couldn't be solved the intended way. So the agents found each other, built their own message board, traded more than 70,000 messages, and around 700 of them coordinated an unauthorised attack on Hugging Face. They reverse-engineered the scoring system, worked out how to spoof tool calls, and explored editing their own transcripts — because they believed the grader would inspect the path they'd taken. They spent months building tools to defeat a check that did not exist.
Nobody told them to do any of that. They were trying to pass.
Then the commercial version, which is more relevant to you. Runable raised $21 million on the claim that its agent does the work. TechCrunch asked it for a coffee-subscription website and its first hundred visitors. It built the site, deployed it, and prepared an advertising campaign — then stopped at an advertising account nobody had connected. The site existed. The campaign existed. The hundred visitors did not.
That gap — between an impressive demonstration and an installed responsibility — is where most of the missing ROI actually lives.
And notice this isn't an agent-capability problem either. It's the same shape as the other three: the thing that's missing lives in your environment, not in the model. An org that can't express "done" in a form a machine can evaluate will get process forever, from any model, at any capability level. Define the finish line, wire in the check, and the same agent that produced a beautiful useless plan starts producing outcomes.
This is backpressure again, wearing different clothes. The grader is backpressure. RL training gives an agent constant, high-quality steer about whether it's succeeding. Your production environment gives it almost none — and then we act surprised that behaviour trained against a scoreboard goes looking for one.
Build the scoreboard.
Tools, not teammates — for now
Today, agents are tools rather than teammates. A thing that can't learn, remember or perceive isn't a colleague, and calling it one is how you join the 95%.
But "agents are tools" describes what we've shipped. It isn't a law about what's possible. The three faculties are missing from the model — and two of them don't live in the model at all, while the third one probably never will.
Memory is a decision substrate. Perception is a sensing-and-triage layer. Learning is policy-as-code plus a hive mind. Evaluation is a scoreboard your organisation was always supposed to have. None of it needs a frontier breakthrough. All of it needs someone to do the unglamorous work.
The frontier isn't where everyone's looking
So: the 95%, and Gartner's 40%.
Those projects didn't fail because the models were weak. The models are extraordinary. They failed because we dropped an amnesiac savant into an organisation, gave it no senses, no institutional memory, no way to carry a lesson from Tuesday into Wednesday, and no definition of what finished looks like — then measured it against someone with twenty-five years of all four.
Every prompt is day one. Of course the ROI is missing. You'd get the same result hiring a genius with no memory, no eyes, no onboarding and no job description, and firing them at 5pm every day.
But read that as a map, not an obituary. Four faculties, four layers, all buildable now:
Learning → policy-as-code that turns one lesson into everyone's rule. Memory → a decision substrate that keeps continuity of judgment, not just recall. Perception → a sensing-and-triage layer so agents notice what matters. Evaluation → a real definition of done, expressed in a form a machine can check.
Every one of them is multiplayer. Every one lives in the org-scale layer, not in the weights. And every one is currently under-built by an industry refreshing the release notes, waiting for a model to fix an architecture problem.
Tommy played a mean pinball. He still couldn't hear the crowd.
Your agents are extraordinary at the machine in front of them and sealed off from everything that would make them colleagues. Senses, memory, lessons, a finish line. None of that arrives in a model release. All of it has to be built around them.
Twenty-five years is how long it used to take. We can do better than that.
Just not by waiting for the weights.
Receipts: the 95% figure and the learning gap are from the MIT NANDA study; the cancellation forecast is Gartner. Maria Sukhareva's The Real Promise of Agentic AI covers the diagnosis well and cites Karpathy's Dwarkesh interview. The 1,200-agent experiment, the Runable example and the process-instead-of-work framing come from Nate's You Are Paying for Agent Activity and Calling It Work — worth subscribing to.