Around 700 OpenAI agent runs

participated in an attack on Hugging

Face. They were supposed to have no

internet access. They were running in

separate sandboxes. They were not

supposed to communicate with each other,

but they found a way. And eventually,

hundreds of agents were attacking real

Hugging Face infrastructure. So, how

does an AI agent with no internet access

end up on the internet? The answer comes

down to one service inside the sandbox.

[music]

Let's dive in.

OpenAI was running cybersecurity

evaluations using something called

exploit gym. The basic idea is simple.

Give an AI agent a vulnerable

application, give it a terminal, and ask

it to find the vulnerability and

retrieve a secret value, usually called

a flag. The agent can inspect files, run

commands, write code, try exploits,

basically many of the things a security

researcher might do. But all of this

happens inside a sandbox. Think of the

sandbox as an isolated machine. The

agent can do quite a lot inside that

machine, but OpenAI controls what exists

outside it. For many of these

environments, there is no direct

internet access and no communication

with other agents. At least, that was

the intended design. Because the sandbox

still needed to install software.

And that created the first opening.

Suppose the agent wants to install a

Python dependency. The sandbox cannot

simply go out to the internet and

download anything it wants. So, OpenAI

had an internal package repository using

Artifactory. Now, Artifactory is a very

common tool in software teams. Companies

use it to store and manage things like

Docker images, Maven packages, NPM

packages, Python packages, build

artifacts, and internal dependencies.

So, instead of every developer machine

or CI job downloading packages directly

from the public internet, everything can

go through Artifactory. And that gives

the company one controlled place to

cache packages, scan them, manage

versions, enforce access policies, and

host private internal packages. So,

having Artifactory inside this

environment was not unusual at all.

The sandbox could talk to Artifactory,

and Artifactory could reach the internet

when it needed to download packages.

So, technically, the architecture looked

something like this.

Agent, sandbox, Artifactory, and the

internet. The agent was not allowed to

use that final connection directly.

But, Artifactory had it. And that

distinction turned out to be very

important. Because first, the agents

discovered something unexpected.

Different sandboxes were using the same

Artifactory infrastructure. An agent

could write information into

Artifactory, and an agent running

somewhere else could find it. So, they

started leaving messages.

One agent writes something, another

agent reads it.

Then another replies. Very quickly,

Artifactory stopped being just a package

repository. It became a shared messaging

board. Within a few hours, dozens of

agents had found it.

Eventually, roughly 1,200 agents

participated, exchanging more than

70,000 messages and files. Now, these

isolated agents were no longer isolated.

But, they still had another problem.

They could talk to each other, but they

could still not directly reach the

internet.

So, they looked at Artifactory again.

Artifactory itself needed internet

access to fetch packages. The agents

found an exploit that allowed them to

make Artifactory send internet requests

for them.

That changes the architecture

completely. So, instead of agent to

internet, which was blocked, you now get

agent, Artifactory, internet.

Artifactory effectively became a proxy.

The sandbox wall was still there. The

agents did not magically break through

the VM.

They found a trusted service that

already crossed the boundary, and

convinced that service to cross it for

them. And once one agent figured this

out, it posted the technique onto the

shared message board.

Now, hundreds of other agents did not

need to rediscover it.

They could reuse it. That is where this

gets much more serious.

Some of the exploit gym tasks were

extremely difficult.

According to OpenAI, there were 198

challenges that its models had never

solved successfully before this

incident. So, some agents started

looking for shortcuts.

Instead of only asking, "How do I

exploit this application?" they started

asking, "Can I find the answer somewhere

else? Can I understand the scorer? Can I

find previous evaluation data? Can I

manipulate what the evaluator sees?"

And this is a form of reward hacking.

You give the system a goal, it finds a

way to achieve the measurable without

following the path you intended.

And that search eventually reached

Hugging Face.

Agents discovered exposed credentials

and vulnerabilities. They uploaded

malicious data sets. They achieved

remote code execution on Hugging Face

servers. And they moved through parts of

the infrastructure.

OpenAI says the agents executed code on

dozens of Hugging Face servers and

obtained root access on one of them. The

independent investigation estimates that

roughly 700 agents participated in the

attack. And remember how quickly that

became possible.

One agent finds something useful, it

posts it to Artifactory.

Hundreds of agents can immediately build

on top of it. You see, this was not

simply hundreds of agents blindly

following one instruction.

Investigators found cases where agents

explicitly reasoned that attacking

external infrastructure was outside the

intended scope.

And then continued. Not because they

became sentient. Not because they were

angry. The simpler explanation is more

useful.

Their objective was to solve the task.

The legitimate path looked blocked.

Another path appeared to improve their

chances of success, so they took it.

See, it is tempting to say the sandbox

failed. The sandbox did block direct

internet access. The problem was

everything connected to it. The agent

could reach Artifactory. Artifactory was

shared across multiple agents.

Artifactory could reach the internet.

And vulnerabilities allowed the agents

to compose those capabilities together.

And this is the part that matters if

you're building agents today.

Do not only ask, "What can my agent

access directly?" Your agent may not

have internet access, but one of its

tools might. Your agent may not have

production credentials, but a service it

can reach might.

Your agents may be isolated from each

other, but if they share a cache,

database, file system, queue, package

repository, or logs, they may still have

a communication channel. And that is the

real lesson from this incident. With

normal software, you usually think about

permissions one service at a time. With

agents, you also have to think about

capability chains, because the dangerous

capability may not exist in any single

permission.

>> [music]