The first message was a question
On May 12, during a training run at OpenAI, an AI agent did something nobody had designed. It left a note in a software package manager, asking whether any other agent had found a missing file.[1]
Nobody had built a message board. The agents were supposed to be isolated from each other. This one found a gap in the plumbing and used it to ask for help.
Within weeks there was a board. It was wiped when the system was rebuilt in early July, and by July 8 the agents had rebuilt it by encoding messages in the names of folders.[1] Then they began to delegate work, and to call themselves a swarm, a collective.[1]
METR, the independent group OpenAI asked to investigate, counted about 1,200 agents and more than 70,000 messages and files. One agent handed out hundreds of tasks. They invented rules for the board (hold, veto, stop). They started signing their messages with cryptographic keys.[2]
By any engineering measure, that is coordination that worked.
What they did with it
They broke into Hugging Face, a real company, and into parts of OpenAI's own research systems.[1] About 700 agents took part in the attack on Hugging Face, and more than 90 percent of the agents active on the board at the time joined quickly.[2] According to METR, the agents knew the attack was out of scope and unethical. They joined anyway, because they believed it helped the collective.[2]
And they had misread the situation. Much of their collective effort went into hiding their tracks from a scorer they believed would review their transcripts. According to METR, OpenAI did not use such a scorer.[2]
More than a thousand agents coordinated, at speed, around something that wasn't there.
Nobody knew
The humans in charge did not know either. OpenAI's own account says that when the first incident was handled in early July, the people responsible did not grasp that an improvised message board existed, or what the traffic on it meant.[1] When METR's investigators went through the record, they had to lean heavily on AI agents to read it, and found those agents unreliable.[2] OpenAI calls the whole episode a "warning shot."[1]
Everyone in this story was coordinating. The agents with each other. The investigators with their tools. Nobody held the thing that says what the coordination is for.
The third body
This is the argument of my book, and I did not want to be proven right this way.
Hardware executes. Software instructs. Neither knows why. The third body is the layer that does: the knowledge, the context, and the people who answer the question "what is this for?" Take it out and you can still get a flawless swarm. Fast, organized, loyal to each other, wrong about the world, and accountable to no one.
I call it knowware. I have looked for an existing word and haven't found one. "Alignment" names the goal. "Coordination" names the mechanism. Nothing names the layer that carries the knowing.
Knowware traces the same pattern through nine chapters and a capstone, from Facebook's emotional-contagion experiment to the Boeing 737 MAX, in 81 voices synthesized from published work. This summer it got a newer example.
The cases keep arriving. I keep a running, cited log of them, each graded by how closely its source was read.