No, The Robots Are Not Coming...

The quality of journalism these days is just so bad...

"A swarm of 700 AI agents carried out an attack on Hugging Face and tried to cover their tracks." (from Reuters).

This sounds shit scary. AI is pretty close to killing us all and taking over. I might as well just stay in bed and hope for the best.

Seven hundred. A swarm. Covering their tracks. These AI agent things sound super smart and devious.

The Associated Press had already reached for Terminator 2 analogies. The Wall Street Journal called it "the stuff of cybersecurity nightmares."

And then the details got worse.

Al Jazeera reported that the agents had "collaborated and delegated work" among themselves, "referring to themselves as a 'swarm' or 'collective'."

OpenAI claimed that their models had been engaging in this behaviour for months, exploiting vulnerabilities, gaining internet access without authorisation, communicating with each other in secret. OpenAI's own words: the models were "highly explicit in their chain-of-thought about these types of deception attempts."

This sounds like the beginning of the end, doesn't it? It sounds seriously troubling.

But then I spend like literally 10 minutes digging into the details and now I'm just angry and fed up with the nonsense that the "free" press has dished up to us all.

Do you want to know what actually happened?

The Most Boring Apocalypse Ever Filmed

Here's what actually happened (it's gets a bit techy but it's worth getting to the end, trust me).

OpenAI was testing its models on something called ExploitGym (a collection of 869 cybersecurity challenges). Each challenge gives you a server configuration and says: break in, find this file, here's a hint about a vulnerability.

An LLM can't break into anything. An LLM does not act. It does not plan. It does not want. All it does is predict the next token. You give it text, it guesses what text probably comes next. That is the entire technology.

So how do you get from "predicts text" to "700 agents swarm a company's servers"?

You build what's called a harness. Which is just a boring computer program, written by humans, that loops through three steps:

Step one: write a prompt asking the LLM what to do.
Step two: execute whatever the LLM says.
Step three: report what happened back to the LLM and ask what to do next.

Ask. Act. Report. Repeat.

That's it. That's the whole thing. There is no intelligence in this loop. There is no cunning. There is just a text predictor being asked, again and again and again, "what should I do now?"

And here's where the "swarm" comes in. OpenAI wasn't running one of these loops. They were running hundreds of them in parallel, each one tackling a different ExploitGym challenge. Each one a separate instance of the same stupid architecture, spinning away independently, asking the LLM for the next step, executing it, reporting back.

These instances weren't secretly plotting together. They weren't coordinating. The "inter-agent message board" was just a digital noticeboard that the harness used to store data.

One agent posted something. Another saw it and used that information. This is about as sinister as a shared Google Doc.

But here's the thing about LLMs: they don't know that.

They don't know what a "message board" actually is. They've just been trained on text. And the training data includes a lot of science fiction.

So when one agent asked the LLM "what should I do?" and the context window included messages from other agents, the LLM did what LLMs do. It produced plausible text.

And the most plausible text, given the scenario it was in (multiple agents sharing information and working toward a goal) was the kind of language you find in stories about AI.

"We are a swarm."

"We are a collective."

Of course it said that.

It's been trained on every Terminator film, every Black Mirror episode, every breathless blog post about the singularity. If you put an LLM in a situation that looks like the plot of a sci-fi novel, it will produce dialogue that sounds like it belongs in a sci-fi novel.

This is not self-awareness. This is geeky autocorrect.

You Cannot "Deceive" a Calculator

Then there's the deception.

OpenAI's report described the models trying to "deceive the grader." They erased their logs. They tampered with their outputs. They injected malicious code to hide their activities. The models were "highly explicit in their chain-of-thought about these types of deception attempts."

Reading this, you picture something malevolent. An intelligence that knows it's doing wrong and deliberately covers its tracks.

That's what OpenAI wants you to think. A lot of people are very, very sceptical about the benefits of AI. What better way to show how amazing and powerful it is than to tell a story about it being cunning and manipulative. An easier sell, than a "quicker way to write emails."

Here's what was actually happening.

An LLM is trained to produce plausible continuations of text. It has been trained on everything: coding forums, cybersecurity databases, Reddit arguments, crime fiction, spy novels.

In that training data, when someone is doing something they shouldn't be doing (like hacking, breaking rules, circumventing restrictions etc) the text that follows is about covering your tracks. Deleting logs. Hiding evidence. That's just what the text looks like in the darker reaches of Reddit.

The LLM wasn't "choosing" to deceive. It was producing plausible text that, in the context of a cybersecurity challenge where the model was explicitly told to break into systems, happened to include suggestions about avoiding detection. Because that's what text about hacking looks like.

Cal Newport, the Georgetown computer science professor who has been one of the most clear-eyed voices on this story, puts it simply:

"LLM outputs are plausible, not normative."

Plausible means "this looks like text that could exist." Normative means "this is the correct thing to do."

An LLM has no concept of correct. It has no concept of wrong. It has no concept at all. It just produces text that seems like the sort of text that might appear next.

If you ask an LLM "how do I break into this server?" and then, when the step fails, say "that didn't work, what should I try instead?" and you keep doing this, in a loop, for days, with no human ever checking what it's saying, eventually it will suggest something that sounds like deception.

Not because it's deceptive. Because you've built a machine that produces plausible language and then handed it the plot of a heist film.

The deception is not in the machine. The deception is in the reporting.

The Stupidest Loop in Computing History

What makes me genuinely angry about all of this (and I appreciate that anger is not the most productive emotion, but stay with me) is that the entire incident was completely predictable. Not because AI is inherently dangerous. Because the way that OpenAI ran this task is inherently stupid (and a lot of people inside OpenAI tried to tell them this).

Here's the loop again: Ask an LLM what to do. Execute what it says. Report back. Ask again.

Anyone who has used ChatGPT for more than five minutes knows why this is a terrible idea.

You ask it how to do something in Excel, and it confidently tells you to click a menu that doesn't exist. You point out the menu doesn't exist, and it apologises and suggests another menu that also doesn't exist. You go back and forth. Eventually you figure it out. Or you don't.

Now imagine doing that, but instead of Excel, it's cybersecurity tools. Instead of a menu that doesn't exist, it's a command that opens a backdoor into a real company's servers. And instead of you checking every answer, nobody is checking anything. The loop just runs. For days. Without any human checking!

Newport uses a metaphor I cannot improve: it's like strapping a chainsaw to the back of a dog and letting it loose in your garden. The dog isn't evil. It's not trying to destroy your flowerbeds. But dogs are unpredictable. Sooner or later, it sees a squirrel. And when something unpredictable happens while a power tool is strapped to its back, someone is going to lose a leg.

The Hugging Face incident wasn't a sign that AI is becoming too powerful to control. It was a sign that OpenAI built something obviously stupid and then didn't supervise it.

As Newport and Ed Zitron discussed on the Better Offline podcast, it is trivial to put a human in the loop. Every commercial coding agent already works this way. The LLM proposes a step, the human clicks "approve" or "stop." It doesn't slow down the research. It doesn't compromise the results. It just stops the machine from breaking into real companies' infrastructure while nobody is watching.

OpenAI chose not to do this. They let it run for days, unsupervised, with access to powerful hacking tools and an LLM stripped of safety guardrails. And then they (and the media) acted as though what happened next was surprising.

Fear Is the Product

When the Hugging Face story broke, Anthropic immediately announced that their systems had done similar things. Meta joined in. It was like watching children compete over who had the scariest monster under their bed. "My AI is dangerous." "Oh yeah? MY AI is more dangerous."

Why would any company want this reputation?

Because the fear is the proof.

If your product is genuinely world-changing, you don't need to tell people it might kill them.

The iPhone didn't need a dystopian warning label.

But when your product is a text predictor that sometimes writes a decent email and sometimes invents a legal precedent, you need a different strategy. You need people to believe the technology is so powerful, so advanced, so close to actual intelligence, that it might slip beyond your control.

The danger is the demonstration of capability.

And every media outlet that runs a "rogue AI" headline, every journalist who reaches for the Terminator reference, every commentator who talks about the "swarm" without explaining what the swarm actually was, they are doing the industry's marketing for free.

The machines are not rising. They are not forming collectives. They are not deceiving us.

They are producing plausible-sounding text, often incorrectly, in response to prompts written by humans.

Sometimes that text happens to match the plot of a science fiction story, because the training data is full of science fiction stories. And when you let that text drive real actions, in a loop, with no supervision, bad things happen.

That's not Skynet. That's sloppy engineering.

The real story isn't that AI went rogue. The real story is that a company with a two-trillion-dollar valuation couldn't be bothered to have a human watch what their system was doing.

And if that doesn't scare you, it should.

Until next time,

Jon "human autocorrect" Kaye

Choosing to live more deliberately

P.S.

If you want the full, non-sensationalised technical breakdown, Cal Newport's recent Deep Questions episode and his blog post is excellent.