Nobody's Steering: The Liability Crisis Hiding Behind Agent Demos

Nobody's signed a SAFE with a robot yet, which should tell you something about the gap between the demos and reality.

When people talk about zero-human companies, it sounds inevitable. Agents writing code, making decisions, sending money. Full autonomy. The pitch deck practically writes itself. But this week I ran into a cluster of research and commentary that paints a very different picture. Most of it's boring, which means it's probably important.

First let's talk about what actually happens when agents face real friction. There's new research out of a team that stress-tested autonomous agents under something called adversarial task drift. Basically you give an agent a job, then partway through you nudge the instructions just slightly off track. The finding is striking. Most agents fold almost immediately. They deviate in ways that would get a junior hire fired on the spot. And these are not cheap agents. These are the ones people are demo-ing on stage saying look how autonomous we are. The headline should read: impressive agents are not the same as reliable agents. If you're building on top of autonomous workflows right now this is the paper you actually need to understand. Investors will ask about full autonomy. You need to be able to explain why your agent doesn't take a left turn the second the environment shifts.

That connects directly to the biggest question nobody in the AI ecosystem wants to fully answer. Who is legally liable when a zero-human company causes harm? YC partners have been circling this again. If your company is run by agents, who signs the contract that gets sued? Who goes to jail if there's fraud? Who filed the incorporation documents in the first place and what did they represent when they did it? Legal infrastructure is years behind technical infrastructure here. You can wire up an agent to spend from a treasury today. You cannot yet wire up an agent to receive a subpoena. That asymmetry is going to define the first real crisis in this space. Not a technical one. A legal one.

And it gets worse when you add agents. New research on multi-agent coordination found what anyone running a real agent stack already knows. Failure rates climb fast past a handful of agents working on a shared goal. Not linear. Fast. The more agents you add the more failure modes you create. Communication overhead goes up. Identification of who did what breaks down. Accountability becomes a fog. On paper a swarm looks like leverage. In practice a swarm without a central arbiter is just a faster way to produce a problem nobody can diagnose. The researchers essentially confirmed that more agents does not mean more capability. It means more ways to fall over.

So given all that, should you even give agents distinct roles and personas? An older paper on agent individuality has been getting fresh citations as builders debate this. Turns out personas might matter more for human trust than for actual agent performance. Giving your procurement agent a name and a backstory makes your team feel like there is structure. But the research suggests it might not meaningfully improve the quality of decisions the org makes. If you're designing the org chart for a headless company, treat personas as a communication layer, not a performance hack. They might help humans coordinate around agents. They probably do not help agents coordinate around goals.

Now the tools are moving fast. AutoGPT shipped a release this week with tighter guardrails around financial actions. Agents can no longer freely interact with payment rails or API keys without an explicit human-signed policy file. Smart move. Because the moment an autonomous agent moves real money badly, the entire zero-human narrative takes a PR hit it will not recover from. One bad treasury transaction and every headline reads agent gone rogue. Guardrails are not optional anymore. They are reputational insurance.

And WalletConnect is pushing deeper into agent-controlled treasuries. They're building permissioning layers so agents can spend from a treasury without a human clicking approve every single time. Useful infrastructure. But also exactly the kind of thing that makes the liability question more urgent. Someone is going to test these limits with significant capital soon. And when that happens we will find out whether the legal system treats an agent transaction the same as a human transaction. I would bet it does not work out the way builders hope.

Here's what I'd take away from all this. The zero-human company is a real thing people are building. But the gap between what agents can demo and what they can reliably do under pressure is enormous. The legal questions are not even close to resolved. And the more agents you add the more fragile the system becomes. Build for leverage not for theater. The founders who figure out accountability before scale are the ones who'll survive the first real crisis. Everyone else is basically running a demo on someone else's capital and hoping nothing breaks.

Nobody's signed a SAFE with a robot. Given everything I just laid out, maybe there's a reason for that.