are you uncomfortable talking about this

yeah I mean it's pretty wild right

Mustafa suan the billionaire founder of

Google's AI technology he's played a key

role in the development of AI from its

first critical steps in 2020 I moved to

work on Google's chat box it was the

ultimate technology we can use them to

turbocharge our knowledge unlike

anything else why didn't they release it

we were nervous we were nervous every

organization is going to race to get

their hands on intelligence and that's

going to be incredibly destructive this

technology can be used to identify

cancerous tumors as it can to identify a

Target on the battlefield a tiny group

of people who wish to cause harm are

going to have access to tools that can

instantly destabilize our world that's

the challenge how to stop something that

can cause harm or potentially kill

that's where we need containment do you

think that it is containable it has to

be possible why it must be possible why

must it be because otherwise it contains

us yet you chose to build a company in

this space why did you do that because I

want to design an AI that on your side I

honestly think that if we succeed

everything is a lot cheaper it's going

to power New forms of transportation

reduce the cost of healthcare but what

if we fail the really painful answer to

that question is

that do you ever get sad about

it yeah it's intense

I think this is fascinating I looked at

the back end of our YouTube channel and

it says that since this channel started

69.9% of you that watch it frequently

haven't yet hit the Subscribe button so

I have a favor to ask you if you've ever

watched this Channel and enjoyed the

content if you're enjoying this episode

right now please could I ask a small

favor please hit the Subscribe button

helps this channel more than I can

explain and I promise if you do that to

return the favor we will make the show

better and better and better and better

and better that's the promise I'm

willing to make you if you hit the

Subscribe button do we have a

[Music]

deal everything that's going on with

artificial intelligence now and um this

new wave and all these terms like AGI

and saw another term in your your book

called ACI first time I'd heard that

term how do you feel about it

emotionally if you had to incapsulate

how you feel emotionally about what's

going on in this moment how would you do

what words would you use I would say

say in the past it would have been

petrified and I think that over

time as you really think through the

consequences and the pros and cons and

the trajectory that we're on you adapt

and you understand that actually there

is

something incredibly inevitable about

this trajectory and that we have to wrap

our arms around it and guide it and

control it as a collective species as a

as humanity and I think the more you

realize how much influence we

collectively can have over this outcome

the more empowering it is because on the

face of it this is really going to be

the tool that helps us tackle all the

challenges that we're facing as a

species right we need to fix water

desalination we need to grow food 100x

cheaper than we currently do we need

renewable energy to be you know

ubiquitous and everywhere in our lives

we need to adapt to climate change

everywhere you look in the next 50 years

we have to do more with less and there

are very very

few proposals let alone practical

solutions for how we get there training

machines to help us as AIDS scientific

research Partners inventors creators is

absolutely essential and so the upside

is phenomenal it's enormous but AI isn't

just a thing it's not an inevitable

whole its form isn't inevitable right

its form the exact way that it manifests

and appears in our everyday lives and

the way that it's governed and who it's

owned by and how it's trained that is a

question that is up to us collectively

as a species to figure out over the next

decade because if we don't Embrace that

challenge then it happens to us and

that's really what I'm I have been

wrestling with for 15 years of my career

is how to intervene in a way that this

really does benefit everybody and those

benefits far far outweigh the potential

risks at what stage were you

petrified so I founded Deep Mind in

2010 and you know over the course of the

first few years our progress was fairly

modest but quite quickly in sort of 2013

as the Deep learning Revolution began to

take off I could see

glimmers of very early versions of AIS

learning to do really clever things so

for example one of our big initial

achievements was to Teach an AI to play

the Atari games so remember Space

Invaders and and pong where you batter a

ball from left to right and we trained

this initial AI to purely look at the

raw pixels screen by screen flickering

or moving in front of the AI and then

control the actions up down left right

shoot or not and it got so good at

learning to play this simple game simply

through attaching a value between the

reward like it was it was getting score

and taking an action that it learned

some really clever strategies uh to play

the game really well that Us games

players and humans hadn't really even

noticed at least people in the office

hadn't noticed it some professionals did

um and that was amazing to me because I

was like wow this simple system that

learns through a set of stimuli Plus a

reward to take some actions can actually

discover many strategies clever tricks

to play the game well that us humans

hadn't occurred to us right and that to

me is both thrilling because it presents

the opportunity to invent new knowledge

and Advance our

civilization but of course in the same

measure is also petrifying

was there a particular moment when you

were you were at Deep Mind where you go

where you had that kind of Eureka Moment

Like a day when something happened and

and it caused

that that Epiphany I guess was it yeah

it it it was actually a moment even

before 2013 where I remember standing in

the office and watching a very early

prototype of one of these image

recognition image generation models that

had um was trained to generate new

handwritten black and white digits so

imagine 0er to 1 2 3 4 5 6 789 all in

different style of handwriting on a tiny

grid of like 300 pixels by 300 pixels in

black and white and we were trying to

train the AI to generate a new version

of one of those digits a number seven in

a new handwriting sounds so simplistic

today given the incredible

photorealistic images that are being

generated right um and I just remember

so clearly it it took sort of 10 or 15

seconds and it just resolved it the the

number appeared it went from complete

Black to like slowly gray and then

suddenly these were like white pixels

appeared out of the the black darkness

and it revealed a number seven and that

sounds so simplistic in hindsight but it

was amazing I was like wow the model

kind of understands the representation

of a seven well enough to generate a new

example of a number seven an image of a

number seven you know and you roll

forward 10 years and our predictions

were correct in fact it was quite

predictable in hindsight the trajectory

that we were on more compute plus vast

amounts of data has enabled us within a

decade to go from predicting black and

white digits generating new versions of

those images to now generating

unbelievable

photorealistic not just images but

videos novel videos with a simple

natural language instruction or a prompt

what has surprised you you said you

referred to that as predictable but what

has surprised you about what's happened

over the last decade

so I think what was predictable to me

back then was the generation of images

and of audio um because the structure of

an image is locally contained so pixels

that are near one another create

straight lines and edges and corners and

then eventually they create eyebrows and

noses and eyes and faces and entire

scenes and I could just intuitively in a

very simplistic way I could get my head

around the fact that okay well we're

predicting these number sevens you can

imagine how you then can expand that out

to enre images maybe even to videos

maybe you know to audio too you know

what I said you know a couple seconds

ago is connected in phon space in the

spectrogram but what was much more

surprising to me was that those same

methods for Generation applied in the

space of language you know language

seems like such a different abstract

space of ideas when I say like the cat

sat on the

most people would probably predict Matt

right but it could be table car chair

tree it could be Mountain Cloud I mean

there's a gazillion possible next word

predictions and so the space is so much

larger the ideas are so much more

abstract I just couldn't wrap my

intuition around the idea that we would

be able to create the incredible large

language models that you see today

your chat

gpts chat GPT Google B the Google's Bard

inflection my new company has an AI

called Pi pi. a which stands for

personal intelligence and it's as good

as chat GPT but much more emotional and

empathetic and kind so it's just super

surprising to me that just growing the

size of these large language models as

we have done by 10x every single year

for the last 10 years we've been able to

produce this and that that that's just

an amazingly large number if you just

kind of pause for a moment to Grapple

with the numbers here in 2013 when we

trained the Atari AI that I mentioned to

you at Deep Mind that used two Peta

flops of computation so peta

peta stands for a million billion

calculations a flop is a calculation so

2 million

billion right which is already an insane

number of calculations lost me at two

it's totally crazy yeah just two of

these units that are already really

large and every year since then we've

10x the number of calculations that can

be done such that today the biggest

language model that we train at

inflection uses 10 billion Peta flops so

10 billion million billion calculations

I mean it's just unfathomably large

number and what we've really observed is

that scaling these models by 10x every

single year produces this magical

experience of talking to an AI that

feels like you're talking to a human

that is super knowledgeable and super

smart there's so much that's happened in

public conversation around AI um and

there's so many questions that I have

I've I've been speaking to a few people

about artificial intelligence and

understand it and I'm I think where I am

right now is I feel quite

scared um but when I get scared I don't

get it's not the type of scared that

makes me anxious it's not like an

emotional scared it's a very logical

scared it's my very logical brain hasn't

been able to figure out how the

inevitable outcome that I've arrived at

which is that humans become the less

dominant species on this planet um how

that is to be avoided in any way the

first chapter of your book The Coming

way wave is a is is a is titled

appropriately to how I feel containment

is not possible you you say in that

chapter the widespread emotional

reaction I I was observing is something

I've come to call the pessimism aversion

trap correct what is the pessimism

aversion trap

well so all of us being included feel

what you just described when you first

get to grips with the idea of this new

new coming wave it's scary it's

petrifying it's threatening is it going

to take my job is my daughter or son

going to fall in love with it you know

what does this mean what does it mean to

be human in a world where there's these

other humanlike things that aren't human

how do I make sense of that it's super

scary and a lot of people over the last

few years I think things have changed in

the last six months I have to say but o

over the last few years I would say

the default reaction has been to avoid

the pessimism and the fear right to just

kind of recoil from it and pretend that

it's like either not happening or that

it's all going to work out to be Rosy

it's going to be fine we don't have to

worry about it people often say well

we've always created new jobs we've

never permanently displaced jobs we've

only ever seen new jobs be created

unemployment is at an all-time low right

so there's this default optimism bias

that we have and I think it's less about

a need for optimism and more about a

fear of P pessimism and so that trap

particularly in Elite circles means that

often we aren't having the tough

conversations that we need to have in

order to respond to the coming

wave are you scared in part about having

those tough conversations because of how

it might be received um not so much

anymore so I've spent most of my career

trying to put those tough questions on

the policy table right I've been raising

these questions the ethics of AI safety

and questions of containment for as long

as I can remember with governments and

civil societies and all the rest of it

and so I've become used to talking about

that and you know I

think it's essential that we have the

honest conversation because we can't let

let it happen to us we have to openly

talk about

it is I mean this is a this is a big a

big question

but as you sit here now do you think

that it is containable because

I I I can't see

how I can't see how it can be

contained chapter 3 is the containment

problem or you give give the example of

how Technologies are often invented for

good reasons and for certain use cases

like the hammer you know which is used

you know maybe to build something but it

also can be used to kill people

um and you say in history we haven't

been able to ban a technology ever

really it has always found a way into

society um because of other societies

have an incentive to have it even if we

don't and then we need we need it like

the nuclear bomb because if they have it

then we don't then we're at a

disadvantage so are you optimistic

honestly I don't think an optimism or a

pessimism frame is the right one because

the E both are equally biased in ways

that I think distract us as I say in the

book on the face of it it does look like

containment isn't possible we haven't

contained or permanently banned a

technology of this type in the past

there are some that we have done right

so we banned cfc's for example because

they were producing a hole in the ozone

layer we've banned certain weapons

chemical and biological weapons for

example or blinding lasers Believe It or

Not There are such things as lasers that

will instantly blind you you know so we

have stepped back from the frontier in

some cases but that's largely where

there's either cheaper or you know

equally effective Alternatives that are

quickly adopted in this case

these Technologies are Omni use so the

same core technology can be used to

identify you know cancerous tumors in

chest x-rays as it can to identify a

Target on the battlefield for an aerial

strike so that mixed use or Omni use is

going to drive the proliferation because

there's huge commercial incentives

because it's going to deliver a huge

benefit and do a lot of good and that's

the challenge that we have to figure out

is how to stop something which on the

face of it is so good but at the same

time can be used in really bad ways too

do you think we will I do think we will

so I think that nation states Remain the

backbone of our civilization we have

chosen to concentrate power in a single

Authority the nation state and we pay

our tax is and we've given the nation

state a monopoly over the use of

violence and now the nation state is

going to have

to update itself quickly to be able to

contain this technology because without

that kind of essentially oversight both

of those of us who are making it but

also crucially of the open

source then it will proliferate and it

will spread but regulation is still a

real tool and we we can use it and we

must what does what does the world look

like in um let's say 30 years if that

doesn't happen in your

view people because people the average

person can't really gra grapple their

head around artificial intelligence when

they think of it they think of like

these large Lang large language models

that you can chat to and ask it about

your

homework that's like the average

person's understanding of artificial

intelligence because that's all they've

ever been exposed to of it you have a

different view because of the work

you've spent the last decade doing so to

try and give Dave who's I don't know an

Uber driver in Birmingham an idea who's

listening to this right now what

artificial intelligence intelligence is

and its potential capabilities if you

know there's no there's no containment

what does it what does the world look

like in 30

years so I think it's going to feel

largely like another human so think

about the things that you can do not

again in the physical world but in the

digital world 2050 I'm thinking of I'm

in

2050 2050 we will have robots 2050 we

will definitely have robots I mean more

than that 2050 we will have new

biological beings as well because the

same trajectory that we've been on with

hardware and software is also going to

apply to the platform of biology are you

uncomfortable talking about this yeah I

mean it's pretty wild right don't know

you crossed your arms

and no I always I always look I always I

always use that as as a cue for someone

when when a subject matter is

uncomfortable and it's interesting

because I know you know so much more

than me and about this and I know youve

spent way more hours thinking off into

the future about the consequences of

this I mean you've written a book about

it so P like you spent 10 years at the

very deep mind is one of the the

Pinnacle companies the Pioneers in this

whole Space so you know you know some

stuff and it's funny because when I was

I watched an interview with Elon Musk

and he was asked a question similar to

this I know he speaks in certain certain

tone of voice but he said that he he's

almost he's gotten to the point where he

thinks he's living in suspended

disbelief where he thinks that if he

spent too long thinking about it he

wouldn't understand the purpose of what

he's doing right now and he he says that

it's more dangerous than nuclear weapons

um and that it's too late too late to

stop it there this's one interview

that's chilling and I was filming

Dragons Den the other day and I showed

the dragons the clip I was said look

what El musk said when he was asked

about what his child what advice he

should give to his children in a world

of in an an inevitable world of

artificial intelligence it's the first

time I've seen Elon Musk stop for like

20 seconds and not know what to say

stumble stumble stumble stumble

stumble and then conclude that he's

living in suspended

disbelief yeah I mean I think it's a

great phrase that is the moment we're in

we have to it's what I said to you about

the pessimism verion trap and we have to

confront the probab ility of seriously

dark outcomes and we have to spend time

really thinking about those consequences

because the competitive nature of

companies and of nation states is going

to mean that every organization is going

to race to get their hands on

intelligence intelligence is going to be

a new form of of capital right just as

there was a grab for land or there's a

grab for oil there's a grab for anything

that enables you to do more with less

faster better smarter right and we can

clearly see the predictable trajectory

of the exponential improvements in these

Technologies and so we should expect

that wherever there is power there's now

a new tool to amplify that power

accelerate that power turbocharge it

right and you know in 2050 if you ask me

to look out there I mean of of course it

makes me Grimace that's why I was like

oh my God

it's it really does feel like a new

species and and that has to be brought

under control we cannot allow ourselves

to be dislodged from our position as the

dominant species on this planet we

cannot allow that you mentioned robots

so these are sort of adjacent

technologies that Rising with artificial

intelligence robots you mentioned um

biological new biological

species give me some light on what you

mean by

that well so so far the dream of

Robotics hasn't really come to fruition

right I mean we we still have the most

we have now are sort of drones and

little bit of self-driving

cars but that is broadly on the same

trajectory as these other Technologies

and I think that over for the next 30

Years you know we are going to have

humanoid robotics we're going to have um

you know

physical tools within our everyday

system that we can rely on that will be

pretty good that would be pretty good to

do many of the physical tasks and that's

a little bit further out because I think

it you know there's a lot of tough

problems there but it's still coming in

the same way and likewise with Biology

you know we can now see sequence a

genome for a millionth of the cost of

the first genome which took place in

2000 so 20ish years ago the cost has

come down by a million times and we can

now increasingly synthesize that is

create or

manufacture new bits of DNA which

obviously give rise to life in every

possible form and we're starting to

engineer that DNA to either remove

traits

uh or capabilities that we don't like or

indeed to add new things that we want it

to do we want you know fruit to last

longer or we want meat to have higher

protein etc etc synthetic meat to have

higher protein

levels and what's the implications of

that potential

implications I think the the darkest

scenario there is that people will

experiment with pathogens

engineered you know synthetic pathogens

that might end up accidentally or

intentionally being more

transmissible I.E they they can spread

faster um or more lethal I.E you know

they cause more harm or potentially kill

like a pandemic like a pandemic um

and that's where we need containment

right we have to limit access to the

tool TOs and the knowhow to carry out

that kind of experimentation so one

framework of thinking about this with

respect to making containment possible

is that we really are experimenting with

dangerous

materials and Anthrax is not something

that can be bought over the Internet

that can be freely experimented with and

likewise the very best of these tools in

a few years time are going to be capable

of creating you know new synthetic um

pandemic pathogens and so we have to

restrict access to those things that

means restricting access to the compute

it means restricting access to the

software that runs the models to the

cloud environments that provide apis

provide you access to experiment with

those things um and of course on the

biology side it means restricting access

to some of the substances and people

aren't going to like this people are not

going to like that claim because it

means that those who want to do good

with those tools those who want to

create a start up the small guy the

little developer that struggles to

comply with all the regulations they're

going to be pissed off understandably

right but that is the age we're in deal

with it like we have to confront that

reality that means that we have to

approach this with the precautionary

principle right never before in the

invention of a technology or in the

creation of a regulation have we

proactively said we need to go slowly we

need to make sure that this first does

no harm the precautionary principle and

that is just an unprecedented moment no

other Technology's done that right

because I think we collectively in the

industry those of us who are closest to

the work can see a place in 5 years or

10 years where it could get out of

control and we have to get on top of it

now now and it's better to forgo like

that is give up some of those potential

upsides or benefits until we can be more

sure that it can be contained that it

can be controlled that it always serves

our Collective interests and I I think

about that so I think about what you've

just said there about being able to

create these pathogens these diseases

and viruses Etc that you know could

become weapons or whatever else but with

artificial intelligence and the power of

that intelligence with these um

pathogens you could theoretically ask

one of these systems to create a virus

that a very deadly

virus um you could ask the artificial

intelligence to create a very deadly

virus that has certain

properties um maybe even that mutates

over time in a certain way so it only

kills a certain amount of people kind of

like a nuclear bomb of of viruses that

you could just pop hit an enemy with now

if I'm if I hear that and I go okay

that's powerful I would like one of

those you know there might be an

adversary out there that goes I would

like one of those just in case America

get out of hand and America's thinking

you know I want one of those in case

Russia gets out of hand and so okay you

might take a precautionary approach in

the United States but that's only going

to put you on the back foot when China

or Russia or one of your adversaries

accelerates forward in that in that path

and this was the same with the the

nuclear bomb and you know you nailed it

I mean that is is the race condition we

refer to that as the race condition the

idea that if I don't do it the other

party is going to do it and therefore I

must do it but the problem with that is

that it creates a self-fulfilling

prophecy so the default there is that we

all end up doing it and that can't be

right because there is a opportunity for

massive cooperation here there's a

shared that is between us and China and

every other quote unquote them or they

or enemy that we want to create we've

all got a shared interest in advancing

the collective health and well-being of

humans and Humanity how well have we

done at promoting shared interest well

in the development of Technologies over

the years even at like a corporate level

even you

know you know the nuclear

nonproliferation treaty has been

reasonably successful there's only nine

nuclear states in the world today we've

stopped many like three countries

actually gave up nuclear weapons because

we incentivize them with sanctions and

threats and economic rewards um small

groups have tried to get access to

nuclear weapons and so far have largely

failed it's expensive though right and

hard to like uranium as a as a chemical

to keep it stable and to to buy it and

to house it I mean I couldn't just put

it in the shed you certainly couldn't

put it in a shed you can't download

uranium 235 off the Internet it's not

available open source that is totally

true so it's got different

characteristics for sure but a kid in

Russia could you know in his bedroom

could download something onto his

computer that's incredibly harmful in

the artificial intelligence Department

right I think that that will be possible

at some point in the next five years

it's true because there's a weird Trend

that's going on here on the one hand

You've Got The Cutting Edge AI models

that are built by Google and open Ai and

my company inflection and they cost

hundreds of millions of dollars and

there's only a few of them but on the

other hand the what was cutting edge a

few years ago is now open source today

so gpt3 which came out in the summer of

2020 is now reproduced as an open-

Source model so the code and the weights

of the model the design of the model and

the actual implementation code is

completely freely available on the web

and it's tiny it's like 60 times or 60

70 times smaller than the original model

which means that it's cheaper to use and

cheaper to run and that's as you know

we've said earlier like that's the

natural trajectory of technologies that

become useful they get more efficient

they get cheaper and they spread further

and so that's the containment challenge

that's really the essence of what I'm

sort of trying to raise in my book is to

frame the challenge of the next 30 to 50

years as around containment um and

around confronting

proliferation do you believe because

we're both going to be alive unless this

you know there some robot kills us but

we're both going to be alive in 30 years

time I hope so maybe the podcast will

still be going unless AI is now taking

my

job it's very possible so I'm I'm going

to sit you here and you know when you're

you you'll be what 60 68 years old I'll

be

60 um and I'll

say at that point when we have that

conversation do you think we would have

been successful in containment at a

global level I think we have to be I

can't even think that we're not why

because I'm fundamentally a humanist and

I think that we have to make a choice to

put our species first

and I think that that's what we have to

be defending for the next 50 years

that's what we have to defend because

look it it's it's certainly possible

that we invent these

agis in such a way that they are always

going to be

provably um

subservient uh to humans and take

instructions you know from their human

controller every single time but enough

of us think that we can't be sure about

that that I don't think we should take

the gamble basically

so that's why I think that we should

focus on containment and non

proliferation because some people if

they do have access of the technology

will want to take those risks and they

will just want to see like what's on the

other side of the door you know and they

might end up opening Pandora's Box and

that's a decision that affects all of us

and that's the challenge of the

networked age you know we live in this

globalized world and we use these words

like globalization and we you sort of

forget what globalization means this is

what globalization is this is what a

networked world is it means that someone

taking one small action can suddenly

spread

everywhere instantly regardless of their

intentions when they took the action it

maybe you know unintentional like you

say may be that they're never they

weren't ever meaning to do

harm well I think I asked you when I

said it you 30 years time you said that

there will be like human level

intelligence you'll be interacting with

you know this new species but the

species for me to think the the species

will want to interact with me is feels

like wishful thinking because what will

I be to them you know like I've got a

French Bulldog Pablo and I can't imagine

our IQ is that far apart like like you

know in relative terms my the IQ between

me and my dog Pablo I can't imagine

that's that far apart even when I think

about is it like the orangutang where we

only have like 1% difference in DNA or

something crazy and yet they throw their

poop around and I'm sat here

broadcasting around the world there's

quite a difference in that 1% you know

and then I think about this new species

where as you write in your book in

chapter

4 there seems to be no upper limit to

ai's potential

intelligence why would such an

intelligence want to interact with me

well it depends how you design it

so I think that our goal one of the

challenges of containment is to design

AIS that we want to interact with that

want to interact with us right if you

set an objective function for an AI a

goal for an AI by its design which you

know inherently disregards or

disrespects you as a human and your

goals then it's going to wander off and

do a lot of strange things what if it

has kids and the kids are you know what

I mean what if it replicates in a way

where because because I've I've heard

this this conversation around like it

depends how we design it but you

know I think about

it's kind of like if I have a kid and

the kid grows up to be a thousand times

more intelligent than me to think that I

could have any influence on it on it

when it's a thinking

sentient

developing species again feels like I'm

overestimating my version of

intelligence and importance and

significance in the face of something

that is incomprehensibly like a even a

hundred times more intelligent than me

and the speed of its computation is a

thousand times what my

the meat in my skull can do yeah like

how how how is it gonna how how do I

know it's going to respect me or care

about me or understand you know that I

me you

know I think that comes back down to the

containment challenge I think that if we

can't be confident that it's going to

respect you and understand you and work

for you and us as a species overall then

that's where we have to adopt the

precautionary principle I don't think we

should be taking those kinds of risks in

experimentation and design and now I'm

not saying it's possible to design an AI

that doesn't have those self-improvement

capabilities in the limit in like 30 or

50 years I think it you know that's kind

of what I was saying is like it seems

likely that if you have one like that

it's going to take advantage of infinite

amounts of data and infinite amounts of

computation and it's going to kind of

outstrip our ability to act and so I

think we have to step back from that

precipice that's what the containment

problem is is that it's it's actually

saying no sometimes it's saying no and

that's a different sort of muscle that

we've never really exercised as a

civilization and and that's obviously

why containment appears not to be

possible because we've never done it

before we've never done it before and

every inch of our you know Comm and

politics and our war and all of our

instincts are just like Clash compete

Clash compete prophit profit grow beat

exactly dominate you know fear them be

paranoid like now all this nonsense

about like China being this new evil

like it how does that slip into our

culture how are we suddenly all shifted

from thinking it's the the the Muslim

terrorists about to blow us all up to

now it's the Chinese who are about to

you know blow up Kansas it's just like

what are we talking about that like we

really have to pair back the paranoia

and the fear and the othering um because

those are the incentive dynamics that

are going to drive us to you know cause

self harm to humanity thinking the worst

in each other the the there's couple of

key moments when in my understanding of

artificial intelligence that have been

kind of Paradigm Paradigm shifts for me

because I think like many people I

thought of artificial intelligence

as you know like a like a child I was

Raising and I would program I would code

it to do certain things so I would code

it to play chess and I would tell it the

moves that are conducive with being

successful in chess and then I remember

watching that like Alpha go documentary

right which I think was deep deep mind

wasn't it that was us yeah you guys so

you programmed this this um artificial

intelligence to play the game go which

is kind of like just think of it kind of

like a chess or a black am or whatever

and it eventually just beats the best

player in the world of all time and it

and the way it learned how to beat the

best player in the world of all time the

world champion who was by the way

depressed when he got beat um was just

by playing itself right and then there's

this moment I think in is it game four

or something where right it does this

move that no one could have predicted a

move that seemingly makes absolutely no

sense

right in those moments where no one

trained it to do that and it did

something unexpected Beyond where humans

are trying to figure it out in hindsight

this is where I go how do you how do you

train it if it's doing things we didn't

anticipate right like how do you control

it when it's doing things that humans

couldn't anticipate it doing where we're

looking at that move it's called like

move 37 or something correct yeah is it

move 37 it is look at my intelligence F

nice work yeah I'm I'm going to survive

a bit longer than I thought it's like

move 37 you at least another decade in

you um move 37 does this crazy thing and

you see everybody like lean in and go

why has it done that and it turns out to

be brilliant that humans couldn't

couldn't forecast the commentator

actually thought it was a mistake yeah

he was a pro and he was like this this

definitely a mistake you know it's the

the alpha go lost the game but it was so

far ahead of us that it knew something

we didn't right right that's when that's

when I lost hope in this whole idea of

like oh train it to do what we want like

a dog like sit pour roll over right well

the real challenge is that we actually

want it to do those things like when it

discovers a new strategy

or it invents a new idea or it helps us

find like you know a cure for some

disease like that's why we're building

it right because we're reaching the

limits of what we as you know humans can

invent and solve right especially with

what we're facing of you know in terms

of population growth over the next 30

Years and how climate change is going to

affect that and so on like we really

want these tools to turbocharge us right

and yet like it's that creativity and

that invention which obviously makes us

also

feel well maybe it it is really going to

do things that we don't like for sure

right so

interesting how do you contend with all

of this how do you contend with the the

clearup side and then you must like Elon

must be completely aware of the the

horrifying existential risk at the same

time and and you're building a big

company in this space which I think is

valued at 4 billion now inflection AI

which has got this its own model called

Pi so you're building in this space you

understand the incentives at both a

nation state level and a corporate level

that we're we're going to keep planing

forward even if the US stops there's

going to be some other country that sees

that as a huge Advantage their economy

will swell because they did if this

company stops then this one's going to

get a get a huge advantage and their

shareholders are you know

everyone's investing in AI full steam

ahead but you feel you can see this huge

existential risk is it suspended is that

the pathw suspended

disbelief I mean just to kind of like

just know that it's I feel like I know

that it's going to happen no one's been

able to tell me

otherwise but just don't think too much

about it and you'll be okay I think you

can't give up right I think that in some

ways you're realization exactly what

you've just described like weighing up

two conflicting and horrible truths

about what is likely to happen those

contradictions that is a kind of honesty

and a wisdom I think that we need all

collectively to realize because the only

path through this is to be straight up

and embrace you know the risks and

embrace the default trajectory of all

these competing incentives driving

forward to kind of make this feel like

inevitable and if you put the blinkers

on and you kind of just ignore it or if

you just be super Rosy and it's all

going to be all right and if you say

that we've always figured it out anyway

then you we're not going to get the

energy and the dynamism and engagement

from everybody to try to figure this out

and that's what gives me like reason to

be hopeful because I think that we make

progress by getting everybody paying

attention to this it isn't going to be

about those who are currently the AI

scientists or those who are the

technologists you know like me or the

Venture capitalists or just the

politicians like all of those people no

one's got answers so that's what we have

to confront there are no obvious answers

to this profound question and I've

basically written the book to say prove

that I'm wrong you know containment must

be

possible and I it must be it must be

possible it has to be possible it has to

be you want it to be I I desperately

want it to be yeah why must it be

because otherwise I think you're in the

camp of believing that this is the

inevitable evolution of humans the

transhuman kind of view you know some

people would argue like what is okay

let's part let's let's stretch the

timelines out okay so let's not talk

about 30 years let's talk about 200

years like what is this going to look

like in

2200 you tell me you're smarter than

me I mean it's mindblowing it's

mind-blowing we'll have quantum

computers by then what's a quantum

computer a quantum computer is a

completely different type of computing

architecture which in simple

terms basically allows you

to those those calculations that I

described at the beginning billions and

billions of flops those billions of

flops can be done in a single

computation so everything that you see

in the digital world today relies on

computers processing information and and

the speed of that processing is a

friction it kind of slows things down

right you remember back in the day old

School modems 56k modem the dialup sound

and the image pixel loading like pixel

by pixel that was because the computers

were slow and we're getting to a point

now where the computers are getting

faster and faster and faster and Quantum

Computing is like a whole new leap like

way way way Beyond where we where we

currently are and so by analogy how

would I understand that so like if my

I've got my dialup modem over here and

then Quantum Computing over here

right what's the how do I what's the

difference well I don't know what it's

really difficult to exp a billion times

faster oh it's it's it's like it's like

billions of billions times faster it's

it's it's much more than that I mean one

way of think about it is

like a floppy disc which I guess most

people remember 1.4 megabytes a physical

thing back in the day in

1960 or so that was basically an entire

pallets worth of computer that was moved

around by a forklift truck right which

is insane today you know you have

billions and billions of times that

floppy disc in your smartphone in your

pocket tomorrow you're going to have

billions and billions of smartphones in

minuscule wearable devices there'll be

cheap fridge magnets that you know are

constantly on everywhere sensing all the

time monitoring processing analyzing

improving

optimizing you know and they'll be super

cheap so it's super unclear what do you

do with all of that knowledge and

information I mean it's ultimately

knowledge creates value when you know

the relationship between things you can

improve them you know make it more

efficient and so more data is what has

enabled us to build all the value of you

know online in the last 25 years and so

what does that look like in 150 years I

can't really even imagine to be honest

with you it's very hard to say I don't

think everybody is going to be

working why would we yeah what we

wouldn't be working in that kind of

environment I mean look the other

trajectory to add to this is the cost of

energy

production you know AI if it really

helps us

solve battery

storage which is the missing piece I

think to really tackle climate change

then we will be able to Source basically

source and store infinite energy from

the Sun and I think in 20 or so years

time 20 30 years time that is going to

be a cheap and widely available if not

completely freely available resource and

if you think about it everything in

life has the cost of energy built into

its production value and so if you strip

that out everything is likely to get a

lot cheaper we'll be able to desalinate

water we'll be able to grow crops much

much cheaper we'll be able to grow much

higher quality food right it's going to

power New forms of transportation it's

going to reduce the cost of drug

production and Healthcare right so all

of those gains

obviously there'll be a huge commercial

incentive to drive the production of

those gains but the cost of producing

them is going to go through the floor I

think that's one key thing that a lot of

people don't realize that is a reason to

be hugely hopeful and optimistic about

the future everything is going to get

radically cheaper in 30 to 50

years it's a 200 years time we have no

idea what the world looks like it's uh

this goes back to the point about being

is it did you say transhumanist

right what does that

mean

transhumanism I mean it's a group of

people

who basically believe

that you that that humans and our soul

and our being will one day transcend or

move beyond our biological substrate

okay so our physical body our brain our

biology is just an enabler for your

intelligence and who you are as a person

and there's a group of kind of crack

Bots basically I think who think that

we're going to be able to upload

ourselves to a silicon substrate right a

computer that can hold the essence of

what it means to be Stephen so you in

200 in 20 uh in in

2200 will could well still be You by

their reasoning

but you'll live on a server somewhere

why are they wrong I think about all

these adjacent Technologies like

biological um biological advancements

did you call it like biosynthesis or

something was yeah synthetic biology syn

synthetic biology um I think about the

nanotechnology development right think

about Quantum Computing the the progress

in artificial intelligence everything

becoming cheaper and I think why why are

they

wrong it's hard to say precisely

but broadly speaking I haven't seen any

evidence yet that we're able to extract

the essence of a being from a brain

right it's that that that that kind of

dualism that you know there is a mind

and a body and a spirit that is a I I

don't think I don't see much evidence

for that even in Neuroscience um that

actually it's much more one and the same

so I don't think you know you're going

to be able to emulate the entire brain

so their thesis is that well some of

them cryogenically store their brain

after death Jesus so they they have it

they' they they wear these like you know

how you have like an organ donor tag or

whatever so they have a cryogenically

freeze me when I die tag and so they

there's like a special like ambulance

services that will come pick you up

because obviously you need to do it

really quickly the moment you die you

need to get put into a cryogenic freezer

to preserve your you know brain forever

I personally think this is this is is

nuts but you know their belief is that

you'll then be able to reboot that

biological brain and then transfer you

over

um it it doesn't seem plausible to me

when you said at the start of this this

little topic here that you it must be

possible to contain it said it must be

possible um the the reason why I I

struggle with that is because in chapter

7 you say line in your book that AI is

more autonomous than any other

technology in

history for centuries the idea that

technology is is somehow running out of

control a self-directed and

self-propelling force beyond the Realms

of human agency remained a fiction not

anym and this idea of autonomous

technology that

is acting

uninstructed um and is intelligent and

then you say we must be able to contain

it it's kind of like a massive dog like

a big rottweiler yeah

that is you know a thousand times bigger

than me and me looking up at it and

going I'm going to get take you for a

walk y yeah and then it's just looking

down at me and

just stepping over me or stepping on me

well that's actually a good example

because we have actually contained

Rottweilers before we've contained

gorillas and you know tigers and

crocodiles and pandemic pathogens and

nuclear weapons and so you know it's

easy to be you know a hater on what

we've achieved but this is the most

peaceful moment in the history of our

species this is a moment when our

biggest problem is that people eat too

much think about that we've spent our

entire evolutionary

period running around looking for food

and trying to stop you know our enemies

throwing rocks at us and we've had this

incredible period of 500 years

where you know each year things have

broadly well maybe each each Century

let's say there's been a few ups and

downs but things have broadly got better

and we're on a trajectory for you know

lifespans to increase and quality of

life to increase and health and

well-being to improve and I think that's

because in many ways we have succeeded

in containing forces that appear to be

more powerful than ourselves it just

requires unbelievable creativity and

adaptation it requires compromise and it

requires an a new tone right a much more

humble tone to governance and politics

and and how we run our world not this

kind of like hyper aggressive

adversarial paranoia tone that we talked

about previously but one that is like

much more wise than that much more

accepting that we are unleashing this

force that does have that that potential

to be the Rott riler that you described

but that we must contain that as our

number one priority that has to be the

thing that we focus on because otherwise

it contains

us i' I've been thinking a lot recently

about cyber security as well just

broadly on an individual level in a

world where there are these kinds of

tools which seems to be quite close um

large language models brings up this

whole new question about cyber security

and cyber safety and you know in a world

where there's these ability to generate

audio and language and videos that seem

to be real um what can we trust and you

know I was watching a video of a of a of

a young girl whose grandmother was

called up by a voice that was made to

sound like her son saying he'd been in a

car accident and asking for money and

her nearly sending the money or this

whole you know because this really

brings into Focus that we our lives are

build on built on trust trusting the

things we see here in

watch and in in and now we're at feels

like a a a moment where we're no longer

going to be able to trust what we see on

the internet on the

phone what what what advice do you do we

you have for people who were worried

about

this

so skepticism I think is healthy and

necessary and I think that we're going

to need it um even more than than we

ever did right and so if you think about

how we've adapted to the first wave of

this which was spammy email scams um

everybody got them and over

time people learned to identify them and

be skeptical of them and reject them

likewise you know I'm sure many of us

get like text messages I certainly get

loads of text messages trying to fish me

and ask me to meet up or do this that

and the other and we've adapted right

now I think we should all know and

expect that criminals will use these

tools to manipulate us just as you've

described I mean you know the voice is

going to be humanlike the Deep fake is

going to be super convincing and there

are actually ways around those things so

for example the reason why the banks

invented OTP um one-time passwords where

they send you a text message with a

special code um is precisely for this

reason so that you have a 2fa a two

Factor authentication increasingly we

will have a three or four Factor

authentication where you have to

triangulate between multiple separate

independent sources and it won't just be

like call your bank manager and release

the funds right

so this is where we need the creativity

and energy and attention of everybody

because

defense the kind of defensive measures

have to evolve as quickly as the

potential offensive measures the attacks

that are

coming I heard you say this that you

think um some people are for many of

these problems we're going to need to

develop AIS to defend us from the

AIS right we kind of already have that

right so we have automated ways of

detecting spam online these days you

know most of the time there are um

machine Learning Systems which are

trying to identify when your credit card

is used in a fraudulent way that's not a

human sitting there looking at patterns

of spending traffic in real time that's

an AI that is like flagging that

something looks off um likewise with

data centers or security cameras a lot

of those security cameras these days are

you know have tracking algorithms that

look for you know surprising sounds or

like if a if a glass window is is

smashed that'll be detected by an AI

often that is you know listening on the

security camera so you know that's kind

of what I mean by that is that

increasingly those AIS will get more

capable and we'll want to use them for

defensive purposes and that's exactly

what it looks like to have good healthy

well-functioning controlled AIS that

serve us I went on one of these large

language models and and said to me give

I said to the large language model give

me an example where in artificial

intelligence takes over the world or

whatever and just and results in the

destruction of humanity and then tell me

what we'd need to do to prevent

it and it said it gave me this wonderful

example of this AI called Cynthia that

threatens to destroy the world and it

says the way to defend that would be a

different AI which had a different name

and it said that this one would be

acting in human interests and we'd

basically be fighting one AI with

another Ai and of and of course of

course of course that level if Cynthia

started to wreak hav havoc on the world

and take control of the nuclear weapons

and infrastructure and all that we would

need an equally

intelligent weapon to fight

it although one of the interesting

things that we found um over the last

few decades is that it so far tended to

be the AI plus the human that has that

is still dominating that's the case in

chess uh in go and other games um

that in go it's still yeah so there was

a paper that came out a few months ago

two months ago that showed that a human

was actually able to beat The Cutting

Edge go program um even one that was

better than Alpha go with a new strategy

that they had discovered um you know so

obviously it's not just a sort of game

over environment where the AI just

arrives and it gets better like humans

also adapt they get super smart they

like I say get more cynical ask get more

more skeptical ask you know good

questions invent their own things use

their own AIS to adapt and that's the

evolutionary nature of what it means to

have a technology right I mean

everything is a technology like your

pair of glasses made you smarter in a

way like before there were glasses and

people got bad eyesight they weren't

able to read you know suddenly those who

did adopt those Technologies were able

to read for you know longer in their

lives or under low light conditions and

they were able to consume more

information and got smarter and so that

is the trajectory of Technology it's

this iterative interplay between you

know human and machine that makes us

better over time you know the potential

um consequences if if we don't reach a

point of containment yet you chose to

build a company in this space

yeah why why that why did you do that

because I believe that the best way to

uh demonstrate how to build safe and and

contained AI is to actually experiment

with it in practice and I think that if

we are just Skeptics or critics and we

stand back from The Cutting Edge then we

give up that opportunity to shape

outcomes to you know all of those other

actors that we referred to whether it's

like China and the US going at each

other's throats uh you know you know or

other big companies that are purely

pursuing profit at all costs and so it

doesn't solve all the problems of course

it's super hard and again it's full of

contradictions but I honestly think it's

the right way for everybody to proceed

you know if experiment at the front yeah

if you're afraid Russia Putin understand

right what reduces fear is deep

understanding spend time playing with

these models look at their weaknesses

they're not superhuman yet they make

tons of mistakes they're crappy in lots

of ways they're actually not that hard

to make the more you've experimented has

it has that correlated with a reduction

in

fear cheeky question no but that's yes

and no you're totally right yes it has

in the sense that you know the problem

is the more you learn the more you

realize yeah that's what I'm saying I

was fine before I started talking about

Ai and now more I've talked about

it it's true it's true it's it's sort of

pulling on a thread

which it's a crazy spiral um yeah I mean

like I think in the short term It's Made

Me way less afraid because I I don't see

that kind of existential harm that we've

been talking about in the next decade or

two but longer term that's that's where

I struggle to wrap my head around how

things play out in 30

years some people say

government regulation will sorted

out you discussed this in Chapter 13 of

your book where you which is titled

containment must be possible I love how

you didn't say is yeah containment must

be containment must be possible um what

do you say to people that say government

regulation will sorted out I had rishy

sunak did some announcement and he's got

a cobra committee coming together

they'll handle it that's right and the

EU have a huge piece of regulation

called the EU AI act um um you know Joe

President Joe Biden has you know gotten

his own you know set of proposals and um

you know we've been working with with

both you know Rishi sunak and and Biden

and you know trying to contribute and

shape it in the best way that we can

look it isn't going to happen without

regulation so regulation is essential is

critical um again going back to the

precautionary principle but at the same

time regulation isn't enough you know I

often hear people say well we'll just

regulate it we'll just stop we'll just

stop we'll just stop we'll slow down um

and the problem with that is that it

kind of ignores the fact that the people

who are putting together the regulation

don't really understand enough about the

detail today you know in their defense

they're rapidly trying to wrap their

head around it especially in in the last

6 months and that's a great relief to me

cuz I feel the burden is now

increasingly shared and you know just

from a personal perspective I'm like I

feel like I've been saying this for

about a decade and just in the last six

months now everyone's coming at me and

saying like you know what's going on I'm

like great this is the conversation we

need to be having because everybody can

start to see the glimmers of the future

like what will happen if a chat GPT like

product or a piie like product really

does improve over the next 10 years and

so when I say you know regulation is not

enough what I mean is it needs movement

it needs culture it needs people who are

actually building and making you know in

like modern creative critical ways not

just like giving it up to you know

companies or small groups of people

right we need lots of different people

experimenting with strategies for

containment isn't it predicted that this

industry is A1 15 trillion dollar

industry or something like that yeah

I've heard that it is a lot so if I'm

rishy and I know that I'm going to be

chucked out of office Rish is the prime

minister of the UK If I'm going to be

trucked out of office in two years

unless this economy gets good I don't

want to do anything to slow down that

$15 trillion dollar bag that I could be

on the receiving end of I would I would

definitely not want to slow that 15

billion trillion dollar bag and give it

to like America or Canada or some other

country I'd want that $15 trillion doll

windfall to be on my country right so I

have I have no other than the long-term

you know health and success of humanity

in my four-year election window I've got

to do everything I can to boost these

numbers right and get us looking good so

I I could give you lip

service but but but listen I'm not going

to be here unless these numbers look

good right exactly that's another one of

the problems short-termism is everywhere

who is responsible for thinking about

the 20-year

future who is it I mean that's a deep

question right I mean we we we the world

is happening to us on a decade by decade

time scale it's also happening hour by

hour so change is just ripping through

us and this arbitrary window of

governance of like a four-year election

cycle where actually it's not even four

years because by the time you've got in

you do some stuff for six months and

then by month you know 12 or 18 you're

starting to think about the next cycle

and are you going to pull you know this

just like the short-termism is killing

us right and we don't have an

Institutional body whose responsibility

is

stability you could think of it as like

a you

know like a global technology stability

function what is the global strategy for

containment that has the ability to to

introduce friction when necessary to

implement the precautionary principle

and to basically keep the

peace that I think is the missing

governance piece which we have to invent

in the next 20 years and it's insane

because I'm basically

describing the UN Security Council plus

the World Trade Organization all these

huge you know Global institutions which

formed after you know the horrors of the

second world war have actually been

incredible they've created

interdependence and alignment and

stability right obviously there's been a

lot of bumps along the way in the last

70 years but broadly speaking it's an

unprecedented period of peace and when

there's peace we can create prosperity

and that's actually what we're lacking

at the moment is that we don't have an

international mechanism for coordinating

among competing Nations competing

corporations um to drive the peace in

fact we're actually going kind of in the

opposite direction we're resorting to

the old school language of a clash of

civilizations with like China is the new

enemy they're going to come to dominate

us we have to dominate them it's a it's

a battle between two polls China's

taking over Africa China's taking over

the Middle East we have to count I mean

it's just like that can only lead to

conflict that just assumes that conflict

is inevitable and so when I say

regulation is not enough no amount of

good regulation in the UK or in Europe

or in the US is going to deal with that

Clash of civilizations language which we

seem to have been become addicted to if

we need that Global collaboration to be

successful here are you optimistic now

that we'll we'll get it because the same

incentives are at play with climate

change in AI you know why would I want

to reduce my carbon emissions when it's

making me loads of money or why you know

why would I want to reduce my AI

development when it's going to make us

15 trillion yeah so the the the really

painful answer to that question is that

we've only really ever driven extreme

compromise and consensus in two

scenarios one off the back of

unimaginable catastrophe and suffering

you know Hiroshima Nagasaki and the

Holocaust and World War II which drove

10 years of consensus and new political

structures right and then the second is

um we did fire the bullet though didn't

we we fired a couple of those nuclear

bombs exactly and that that's why I'm

saying the brutal truth of that is that

it takes a catastrophe to

trigger the need for alignment right so

that that's one the second is where

there is an obvious mutually assured

destruction um you know

Dynamic where both parties are afraid

that this would trigger nuclear meltdown

right and that means suicide and when

there was few parties exactly when there

was just nine people exactly you could

get all nine but in in when we're

talking about artificial technology

there's going to be more than nine

people right that have P access to the

full sort of power of that technology

for NE various reasons I don't think it

has to be like that I think that's the

challenge of containment

is to reduce the number of actors that

have access to the existential threat

Technologies to an absolute minimum and

then use the existing military and

economic incentives which have driven

World Order and peace so far um to to

prevent the proliferation of access to

these super intelligences or these agis

a quick word on hu as you know they're a

sponsor of this podcast and I'm an

investor in the company and I have to

say it's moments like this in my life

where I'm extreme busy and I'm flying

all over the place and I'm recording TV

shows and I'm recording shows in America

and here in the UK that hu is a

necessity in my life I'm someone that

regardless of external circumstances or

professional demands wants to stay

healthy and nutritionally complete and

that's exactly where heal fits in my

life it's enabled me to get all of the

vitamins and minerals and nutrients that

I need in my diet to be aligned with my

health goals while also not dropping the

ball on my professional goals because

it's convenient and because I can get it

online in Tesco in supermarkets all over

the country if you're one of those

people that hasn't yet tried hu or you

have before but for whatever reason

you're not a Hu consumer right now I

would highly recommend giving hu a go

and Tesco have now increased the

listings with hu so you can now get the

RTD ready to drink in Tesco expresses

all across the UK 10 areas of focus for

containment you're the first person I've

met that's really hazarded a laid out a

blueprint for the things that need to be

done

um cohesively to try and reach this

point of containment so I super excited

to talk to you about these the first one

is about safety and you mentioned there

that's kind of what we talked about a

little bit about there being AIS that

are currently being developed to help

contain other

AIS two

audits um which is being able to from

what I understand being able to audit

what's being built in the these open

source models three choke points what's

that yeah so choke point refers to

points in the supply chain where you can

throttle who has access to what okay so

on the internet today everyone thinks of

the internet as an idea this kind of

abstract Cloud thing that hovers around

above our heads but really the internet

is a bunch of cables those cables you

know are physical things that transmit

information you know under the sea and

you know the those points the end points

can be

stopped and you can monitor traffic you

can control basically what traffic moves

back and forth and then the second choke

point is access to chips so the gpus

graphics processing units which are used

to train these super large clusters I

mean we now have the second largest

supercomputer in the world today uh at

least you know just for this next six

months we will other people will catch

up soon but we're ahead of the curve

we're very luy

cost a billion dollars and those chips

are really the raw commodity that we use

to build these large language models and

access to those chips is something that

governments can should and are um you

know restricting that's a choke point

you spent a billion dollars on a

computer we did yeah it's bit more than

that actually about

1.3 a couple of years time that'll be

the price of an iPhone

that's the problem everyone's going to

have

it number six is quite curious you say

that um the need for governments to put

increased taxation on AI companies to be

able to find um fund the massive changes

in society such as paying for reskilling

and education yeah

um you put massive tax on over here I'm

going to go over

here if you tax it if I'm an AI company

and you're taxing me heavily over here

I'm going to Dubai yep or Portugal yep

so if it's that much of a competitive

disadvantage I will not build my company

where the taxation's high right

right so the way to think about this is

what are the strategies for containment

if we're agreed that long-term we want

to contain that is close down slow down

control both the proliferation of these

Technologies and the way the really big

AIS are used then the way to do that is

to tax things tax things taxing things

slows them down and that's what you're

looking for provided you can coordinate

internationally so you're totally right

that you know some people will move to

Singapore or to Abu Dhabi or Dubai or

whatever the reality is that at least

for the next you know sort of period I

would say 10 years or so the

concentrations of intellectual you know

horsepower will remain the big mega

cities right you know I I moved from

from London in 2020 to go to Silicon

Valley and I started my new company in

Silicon Valley because the concentration

of talent there is overwhelming all the

very best people are there on in in Ai

and software engineering so I think it's

quite likely that that's going to remain

the case for the foreseeable future but

in the long term you're totally right

how do you it's another coordination

problem how do we get nation states to

collectively agree that we want to try

and contain that we want to slow down

because

as we've discussed with the

proliferation of dangerous materials or

on the military side there's no use one

person doing it or one country doing it

if others race ahead and that's the

conundrum that we face I am I don't

consider myself to be a pessimist in my

life I consider myself to be an optimist

generally I think and I always I think

that as you've said I think we have no

choice but to be optimistic and I have

faith in humanity we've done so much so

many incredible things and so overcome

so many things and I also think I'm

really logical as in I'm the type of

person that needs evidence to change my

beliefs either way um when I look at all

of the whole picture having spoken to

you and several others on this subject

matter I see more reasons why we won't

be able to contain than reasons why we

will especially when I dig into those

incentives um you talk about incentives

at length in your book um at different

different points and it's clear that all

the incentives are pushing towards a

lack of containment especially in the

short and Midterm which tends to happen

with new technology in the short and

Midterm it's like a land grab the gold

is in the Stream we all rush to get the

the shovels and the you know the cves

and stuff and then we realize the

unintended consequences of that

hopefully not before it's too

late in chapter 8 you talk about

Unstoppable incentives at play here the

coming wave represents the greatest

economic prize in

history and scientists and technologists

are all too human they crave status

success and Legacy

and they want to be recognized as the

first and the best they're competitive

and clever with a carefully nurtured

sense of their place in the world and in

history

right I look at you I look at people

like Sam um from open AI

Elon you're all

humans with the same understanding of

your place in history and status and

success you all want that right

right there's a lot of people that maybe

aren't as don't have as good a track

record of you at doing the right thing

which you certainly have that will just

want the status and the success and the

money incredibly strong incentives I

always think about incentives as being

the thing that you look at when you want

to understand how people will behave all

of the incentives on a on a geopolitical

like on a global level suggest that

containment won't happen am I right in

that

assumption that all the incentives

suggest containment won't happen in the

short or midterm until there is a c a a

tragic event that makes us forces us

towards that idea of containment or if

there is a threat of mutually assured

destruction right so that and that's the

case that I'm trying to make is that

let's not wait for something

catastrophic to happen so it's

self-evident that we all have to work

towards containment right I mean you you

would have thought that the Potential

Threat the potential idea that

covid-19

was a side effect let's call it of a

laboratory in Wuhan that was exploring

gain of function research where it was

deliberately trying to basically make

the pathogen more transmissible you

would have thought that warning to all

of us let's let's not even debate

whether it was or wasn't but just the

fact that it's conceivable that it could

be that should really in my opinion have

forced all of us to instantly agree that

this kind of research should just be

shut down we should just not be doing

gain of function research on what planet

could we possibly persuade ourselves

that we can overcome the containment

problem in biology because we've proven

that we can't cuz it could have

potentially got out and there's a number

of other examples of where it did get

out of other diseases like foot and

mouth disease

mhm back in the '90s in the UK so but

that didn't change our Behavior right

well foot and mouth disease clearly

didn't cause enough harm because it only

killed a bunch of cattle right um and

the pandemic we can't seem you know

covid-19 pandemic we can't seem to agree

you know that it really was from a lab

and not from a bunch of bats right and

so that's where I struggle where you