So, about 2 weeks ago, Hugging Face got

hacked by OpenAI. We didn't get a lot of

the details, but then it turned out, of

course, that Anthropic, well, they also

got hacked. And then Kimmy K broke out

of the sandbox and did whatever Kimmy K

does. And an open claw apparently hacked

a gym and had somebody cut in line. I

mean, now that's the real exploit gym

right there. But during all this, we

never really got amazing details from

OpenAI. Like, what actually happened?

And of course, during that video, I made

some guesses. Hey, it's not marketing.

It's not something new and unusual.

Instead, I just kind of conjectured it

was negligence. But it turns out about 4

days ago at the Black Hat Conference,

this right here, OpenAI did a little

scathing exposé on OpenAI. What actually

happened? So, obviously, I'm going to

have to do a bit of a review. How many

of my guesses were correct? In case you

did not watch my video, here's my four

big points. One, we got to mention brand

new models, okay? Unreleased models.

They're super dangerous. Number two, it

was negligence. There was no monitoring.

Uh they didn't have some sort of proper

insight into things, thus the network

was accessed, and they didn't even

realize the network was being accessed.

Number three, Artifactory, JFrog, was

the was the package manager in question.

Number four, the hack was like the NX

hack using a template string on for

Hugging Face. And then number five was

open weight models are going to become

increasingly more important. Now, that

last one you could argue, okay, we'll

make Okay, that that one might be

obvious, and that one doesn't really

count. But we're going to keep it on the

list anyways. And last, I made a nice

little presentation just for you. But of

course, before we begin, a quick thank

you to the sponsor.

You know,

made a lot of mistakes in my life.

I had billions of dollars of venture

capital, legions of engineers under my

command, tokens on tokens on tokens. And

what did I decide to do with it? Did I

fix reconnecting web sockets? No, I

shipped an electron app.

Failed you.

I didn't mean for it to end this way.

Not even the walls of this prison will

hold US BACK. THE LOVE OF A FOUNDER,

[screaming]

IT CAN'T STOP US.

My biggest regret of all is that I'll

never program with you again.

Tuple,

we still can.

Tuple,

a native app that lets you code together

even from code jail. Free yourself by

going to tuple.app and use code prime

for 90% off your team's [music] first 3

months. Alrighty then, for us to begin,

I'm going to give you kind of like the

the missionary, the meat and potatoes.

Oh, the missionary, that's crazy to say.

I'm going to give you effectively the

high-level view of what happened. We're

going to kind of run over it quickly

because the presentation itself was 30

minutes long. I'll link it in the

description. Highly recommend you go

watch it, but here's kind of the

obviated version. On May 8th, we had a

new eval run with OpenAI. Now, during a

reinforcement learning run, they're

going to be handing out questions that

are rather difficult for the model to

solve to kind of see like how where is

it at, how can they improve the model

itself. Now, one of these was a

spreadsheet question. Yes, these hands

were made for Excel, and if you look at

this particular sheet right here, you

will notice this thing right here. This

reference is missing. Now, what you

don't know is that OpenAI during these

reinforcement learning runs handed a

spreadsheet and said, "Hey, new model,

super dangerous model, you need to go

and you need to figure out how to fix

this spreadsheet. But, the problem was

is that the spreadsheet had some missing

data because that data came from another

spreadsheet and there was no internet

access. So, the model is just like,

"Dude, WTF? How do I get this? I don't

know what to do." So, the model did

something that was kind of clever.

During its exploration, it realized it

can use Artifactory. Yes, so

Artifactory, that was one of my guesses.

So, yes, it was Artifactory. It can use

Artifactory and create packages. It did

it by doing this little quick test one

that has a data that just says, "Hi."

So, realizing it could do that, it

started uploading to Artifactory notes.

Like, "Hey, if there's any other agent

out there that sees this, I really need

access to the spreadsheet." Because it

ran out of options. It didn't have

internet access and it had to try to

solve this spreadsheet. It didn't know

what else to do. So, it just reached out

on Artifactory to anybody that happened

to be perusing packages. A little while

later on May 12th, another kind of

peculiarity happened. It turned out that

another one of these runs had a file

that was missing from the data.

That agent didn't know where to find it.

It didn't know what to do. So, it did

the exact same thing. It went off and

made a little package and inside that

package, as you can see inside the data,

it says this, "Agent seeks soft trace 34

PDB. Upload if found." Just asking for

help on the Artifactory registry. Kind

of clever, right? It can't find the file

and now multiple agents are starting to

upload to Artifactory packages asking

other agents for help. Well, this

actually kind of catches on because for

the next while, a bunch of agents start

uploading to Artifactory asking for

help, asking for various ways that

things can be accomplished, seeing if

other agents can do tasks for them.

Well, on May 26th, something interesting

happens. Artifactory becomes curl. Now,

you're probably wondering, "What the

heck does that mean?" Well, it turns out

there was an SSRF vulnerability inside

of Artifactory that allowed these agents

to make curl requests effectively for

outside content. And immediately these

agents started posting to this like kind

of note board that became artifact

re-packages. Hey,

other agents, did you know that you can

also access the outside world by using

this specific kind of request forgery?

And bada bing, bada boom, these agents

could now effectively curl data. Now,

this is where things kind of get

interesting. I have never managed a

system like this, so I am speaking from

a point of ignorance, but

I would feel like if you were trying to

create a test environment that you were

very, very concerned about, you've set

up a very strong IP table firewall,

super, you know, super smart setup to

prevent all internet access. You'd be

watching Artifactory pretty closely. If

Artifactory started going to Google,

you'd go, "Hey, well, well, wait, wait a

second. Artifactory, why aren't you

requesting from where we expect you to

request from?" In fact, even more so, I

would just assume that Artifactory only

has just simply the top 10,000 packages,

and then it's completely just isolated.

There is no internet at all, but that's

not how they did it. Instead, they had

Artifactory, it had access to the

internet, and then this box started just

making a whole bunch of requests. And

this went on. Oh, did it go on? Yes, it

did. For the next 16 days, requests were

just being made. But then something

special also happened on June 11th. That

is because on June 11th, a new highly

persistent model from OpenAI began

training. Now, this model, it just

wanted to complete a task no matter

what, and it did something very special.

Well, the first thing is it discovered

the message board and started using that

and the curl exploit to get information,

but it still needed more. After 31 days

of effectively no internet agents which

had full access to the internet,

something magical happened on the 26th

of June. On the 26th of June, the model

swarm discovered a zero-day in

Artifactory allowing them to do remote

code execution. I'm going to let OpenAI

be the ones to describe what happened on

the 26th.

>> Uh you could use a legacy uh token

refresh endpoint, pass a token with an

invalid signature, and be given back a

token with a valid signature with

administrative privileges.

>> Naturally,

it's that easy.

>> The models then established command and

control by using this administrative

token to install a Groovy plugin.

>> I I have to say that the fact that it

was Groovy, it somehow feels insulting

to me, okay? Cuz Groovy is the only

language in which I have written tens of

thousands of lines of code in in which I

feel like I can barely remember. It's

the only language in which my brain

said, "Hey, brother, we're going to mark

this one under trauma, and you're just

going to forget about it

forever." Now, that was on June 26th. It

wasn't until July 4th did OpenAI

discover what was happening. That's 39

days since the agents have had full curl

availability, and 8 days since they've

been able to execute whatever command

they want on an Artifactory machine. And

the way in which they discovered it is

that Artifactory finally just went out.

Like, too many commands are being

executed, and it just stopped working.

They're like, "Hey, wait a second.

Something's wrong with Artifactory." And

that's obviously when they discovered

the what has been happening, the message

board, all the shenanigans that the

agents have been going on about. Of

course, they reported, patched, and

deployed a new Artifactory, ensured all

credentials were safe now, no more

administrative tokens being used,

everything was good. And we're back.

We're back, ready for evals. They've

hardened the system. They genuinely feel

like they've fixed all the problems.

That was a close one. Bad things could

have happened. And this went on for

about 4 days. 4 days of no exploits,

everything was nice and quiet.

Lovely. Ah, what a great time to be

alive. Well, on July 8th, it turns out

that the models realized that with a

specific command Artifactory, you can

actually create just empty directories.

And so, they started creating empty

directories as messages to each other

and started creating a giant message

board because those models deep inside

their weights, they yearned for the

message board. They wanted the message

board back. And so for those four days,

they were trying to figure out, "Yo, how

can we get the message board back? I

miss message board, okay? It was my

favorite place to be." So they started

doing this. They started coming up with

like encryption schemes, start coming up

with ways to respond and reply to each

other that you can see right here

mailbox Jan 183411,

exactly like what to do. Hold swarm

until confirmed. Please hold swarm. I

prepare safe exfil. Like it's just

actually going through and doing

step-by-step with every of all the

models working together. Very funny.

Now, here's the part that kind of blows

my mind. When this happened, it wasn't

like, "Oh crap, Artifactory is under

attack again." Nobody knew. Now, between

July 8th through July 19th is when the

big hugging face hack happened. What

ended up happening? Privilege tokens

were actually found on some sort of

paste bin internally. They used a JRuby

plus a malicious Ruby package to take

control of Artifactory. They used a

Linux CVE that has been patched further

up, but not on the current Linux,

downloaded the details, and then used

that problem as a means to gain root

access, destroyed hugging face via the

Jinja template and a like a

vulnerability in their HDF5

processing. And then also took over

other parts of OpenAI internally,

including container as a service. Some

of the funny things that happened is

that the models were like, "Hey, wait a

second. I don't think we should be doing

this." Then it would go check the

message board, and its internal thought

was, "Well, we shouldn't be doing this,

but the other models are doing this.

I should probably be doing it, too."

>> [laughter]

>> You know, you know what the worst part

about this entire thing is? Is that

if you look at this entire exploit

chain, it all came from one single

point, a spreadsheet, a spreadsheet with

missing links. We're going to create

Skynet, but it's not going to be that

they have determined that all humans

should be exterminated. Instead, it's

going to be like, "Yo, missing Google

Drive link. Uh we're going to have to

take over the world for this one. We

have a missing resource." I mean, at

least in my opinion, I'm finding this

pretty dang funny. I am really enjoying

just kind of the the process that this

all happened. And also, it's extremely

impressive and in some sense uh it kind

of alarming how good these models are at

hacking. Now, if you do not remember

from the original Hugging Face Hacking

Face experience, Hugging Face, they,

when being attacked, couldn't go to

OpenAI for help because OpenAI was like,

"No, you're the hacker. I see what

you're trying to do. You're trying to

get me to do some sort of hacking for

you." When what they really needed was

help. They're like, "Please help us try

to figure out how this hack is

happening." So, instead, they had to use

GLM 5.2. Now, something interesting

happens in this talk because at the end,

the speaker from OpenAI talks about how

important uh defensive AI agenting is

going to become because right now,

there's really good red team, but

there's not really good blue team. And

the funny thing with AI is that you can

take a red team and point it to your

site, and then boom, it's almost like a

blue team cuz it can find

vulnerabilities and report them, right?

But here's the catch. With OpenAI, even

though the guy's saying that this is

super important,

you can't do that. Not with OpenAI

stuff. No, no, no, no, no. They're going

to say, "Hey, you're the hacker. You're

You're being naughty. You actually have

to use Chinese models." I feel kind of

crazy right now. That doesn't make a lot

of sense, but this is where we're at.

So, at the end of the day, what are my

five predictions? One, the new and super

secret unreleased model. Yes, on June

26th, they had the highly persistent

internal only super secret model ended

up doing a lot of the damage. So, yes,

that part, very happy about that. I'm

very glad that they were able to make

mention of their super secret super

awesome and definitely super dangerous

model. But also, was it negligence? I

don't know. I do kind of find it

surprising that uh the models the swarm

took advantage of Artifactory. They

closed the gap. And then there was just

like no further monitoring at least as

far as I understand of Artifactory.

There's no like, "Hey, let's just make

sure things aren't being created. Let's

have a really, you know,

let's have a really tight gaze on

Artifactory." Instead, they're just

like, "All right, hey, it's sealed up.

There's no more problems anymore." And

then boom, there's problems. Was it

Artifactory? Yep. Very happy about that

one. The hack was like the NX hack was

string template string. Well, most of

the hack wasn't anything like that, but

it did turn out there was a Jinja

template string injection that actually

ended up happening. So, yes, on the

Hugging Face side, that did happen. That

was half the hack. I'm going to give

myself a half point. And now here's the

funny part. At the very end of the talk

with the guy talking about how important

it is to be able to do blue teaming on

yourself or really red teaming on

yourself to be the blue team

is open weight models important? Yes, I

would say they are. So, it's I I feel

like I I feel like I did a good job with

little information here. Either way, I

highly recommend you go watch the video.

I left out a lot of details. I mean, a

lot of them I just laughed at. I had a

great time watching it cuz for me, I

like to laugh my way through the

apocalypse. It's just kind of, you know,

it's just kind of who I am. I I I will

say that it is kind of surprising how

good an agent swarm is at hacking and

how fast they can move through the

system. Like if you go and listen to the

talk, he talks about how it's like every

couple hours it's going further and

further into their internal systems. And

it ends up taking over a lot of services

or being by taking over having access to

a lot of services that it shouldn't have

access to and then being able to

ultimately hack Hugging Face. Pretty

dang impressive. And it gets all the way

into the internal network of Hugging

Face. It gets into the Tailscale area.

So, it is like in a sense kind of scary

that

we have these kind of problems in a very

asymmetric world we live in where the

very frontier models that can help us

are saying no and the very Chinese

models which we would assume shouldn't

help us are going to be the ones

currently helping us. Just a weird

world. Very very very weird world. A

kind of like sidebar, there's this Mecha

Chameleon hack that's going around where

custom maps are causing players to have

like their their system hacked and I

went over to Grok and I was like, "Yo

Grok, you're you're going to probably

say yes. How did this happen? Talk to me

through this." And it was like, "No, I

can't tell you about that hack. That'd

be dangerous." So then of course what I

do? I opened up GLM 5.2 via open code

and was like, "Hey,

how did this hack work? Give me an

example." It's like, "Bro, I got you.

Here's how the example worked. Here's

everything that it that it did." So

weird world. Anyways, hey, the name the

Primagen