What have you learned about AI from

your trip to China and thinking about

China, speaking to Chinese

people, whether they're technologists or

otherwise? Like, what have you learned

during or since that trip?

>> Back in March, before my book was

published in the US,

I went to China because the Chinese are

faster at everything, including

publishing books.

And uh my publisher brought me out there

and basically, you know, took me around

four cities, eight days, meeting with AI

leaders both in academia and big

companies like Huawei and Hikvision and

Ant Group.

And the thing which was surprising

was the extent to which people brought

up the issue of AI safety.

And I say that was surprising because my

friends who had done AI policy in the

Biden administration

had primed me to expect that there would

be no mention of safety in China, that

they basically didn't care about it,

that

you know, the muscle memory that we have

in the West

of technology being dangerous, you know,

the atom bomb experience, the Cuban

missile crisis,

our ambivalence about technology is not

shared in China, where their idea of

catastrophe is sort of like, you know,

the Cultural Revolution, it's some

political thing that goes wrong. And

conversely, technology has been part of

their amazing growth story in the last

25 years, which they are rightly proud

of and delighted by. So, they love

technology, right? So,

when the Biden team tried to meet with

the Chinese and talk about AI safety,

they got nowhere and they decided it was

impossible to even talk to them about

some sort of non-proliferation treaty

for AI.

But when I went there, I found they did

talk about safety kind of unprompted.

And this led me down this track of

arguing over the last couple of months

that the door is actually open

to a dialogue with China

about preventing bad guys doing bad

stuff

with AI.

Because they don't want the internet to

be crashed by some cyber hacker who has

the tool.

They don't want bio weapons, they don't

want chemical weapons, they want none of

that. They love regulating the internet,

right? So we have a shared interest with

the Chinese in preventing this

proliferation of risk

from going nuts.

And as I thought about it, you know, the

kind of Cold War analogy

came to seem more and more opposite,

right? So if you look back at the story

of nuclear weapons, there were two kinds

of danger.

First danger is you have a nuclear war

between the Soviet Union and the United

States.

But that was contained by balance.

Two superpowers, they both have the

weaponry.

They have mutually assured destruction,

so there's no war.

Then there's another kind of risk, which

is that other random rogues, whether

it's criminals, terrorists, rogue

states,

get the stuff and they do bad stuff, and

it's much harder to deter that because

it's a multipolar game.

And so deterrence doesn't work so

elegantly.

And so the way it was dealt with in the

Cold War was that in 1956 there was the

agreement on the International Atomic

Energy Agency.

And in 1968 the Non-Proliferation Treaty

kind of enforced compliance with the

IAEA

such that you could get civilian nuclear

power

if you were a non-nuclear state, but you

had to submit to the rules and be

inspected and show that you were not

using

the enriched nuclear material to build a

weapon, right?

And so I think the same analogy could be

applied to AI. We're going to have

parity roughly with China. We'll both

have powerful AI. Hopefully deterrence

prevents war breaking out.

But at the same time, we don't want open

weight models that can be freely

downloaded by anybody who wants to fall

into the hands of criminals and

terrorists

who can then use it to hold us hostage.

And we have a joint interest in that.

And you know, when my friends from the

Biden team or even from the current

administration say,

"Well, you can't talk to China about

safety. They don't care." I say, "That's

not true." And they say, "But it's

really hard. They don't stick by their

commitments." And I go, "You think

Nikita Khrushchev in [snorts] the Soviet

Union was easy to negotiate with? He was

the guy who put missiles in Cuba and

went to the UN and banged his shoe on

the table and said, "We will bury you."

I mean, he was a tough guy to talk to,

but we did talk to him and we got the

non-proliferation treaty agreed.

And I think we need to do the same thing

again now.

>> Where do you stand on

your thinking about chip export?

>> So, when the chip export controls were

announced,

which was October of 2022, right before

ChatGPT,

I supported those controls quite loudly.

I wrote a very long piece in the

Washington Post saying that if we could

stop China getting frontier models by

depriving them of frontier chips,

I was all in favor of that because of

the strategic advantage for the US. I

mean, I work at the Council on Foreign

Relations.

We do geopolitics and national security

all day long.

And I'm all in favor of US power.

But I have to say that, you know, 3 and

1/2 years later,

we haven't actually achieved that

enormous advantage over China in terms

of the models. Based on the best

studies, we're kind of 8 months ahead in

terms of where the frontier model is,

like our frontier model versus their

frontier model.

And then if you adjust for the speed

with which the model gets turned into an

application,

probably that gap shrinks, and it may

even be nonexistent.

So, however you slice that, the basic

bottom line is we both have strong

models, and the chip export controls

have not delivered what I hoped would be

the big advantage.

And so, I'm not against keeping the

controls on

if we think that maybe as the compute

demands of bigger and bigger models

bite,

the chip controls will bite more, and

maybe we get a bigger advantage next

year or something.

But I don't want the chip controls to

get in the way of discussion with the

Chinese about where we have a shared

interest,

which is in controlling open weight

models

and preventing the bad stuff falling

into the hands of the bad guys.

I would prioritize

collaboration with China, and if that

meant, you know, loosening up a little

bit on the export controls, I would be

okay with that.