for the past couple of months I've been

having this strange experience where

person after person independent of each

other from AI Labs from government has

been coming to me and saying it's really

about to happen artificial general

intelligence AGI AGI AGI that is really

the Holy Grail of ai ai systems that are

better than almost all humans at almost

all tasks and before they thought you

know may it take 5 or 10 years 10 or 15

years now they believe it's coming

inside of 2 to 3 years a lot of people

don't realize that AI is going to be a

big thing inside Donald Trump's second

term and I think they're right and we're

not prepared in part because it's not

clear what it would mean to prepare we

don't know how labor markets will

respond we don't know which country is

going to get there first we don't know

what it will mean for war we don't know

what it will mean for peace and as much

as there is so much else going on in the

world to cover I do think there's a good

chance that when we look back on this

era in human history this will have been

the thing that matters this will have

been the Event Horizon thing that the

world before it and the world after it

were just different worlds one of the

people reached out to me is Ben Buchanan

who was the former special adviser for

artificial intelligence in the Biden

White House he was at the nerve center

of what policy we have been making in

recent years but there's now been a

profound changeover in administrations

and the new Administration has a lot of

people with very very very strong views

on AI so what are they going to do what

kinds of decisions are going to need to

be made and what kinds of thinking do we

need to start doing doing now to be

prepared for something that virtually

everybody who works in this area is

trying to tell us as loudly as they

possibly can is coming as always my

email as reclin show NY

times.com Ben M Canan welcome to the

show thanks for having me so you give me

a call after the end of the B

Administration and I got a call from a

lot of people in the B Administration

who wanted to tell me about all the

great work they did and you sort of seem

to want to warn people about what you

now thought was coming what's coming I

think we're going to see extraordinarily

capable AI systems I don't love the term

artificial general intelligence but I

think that will fit um in the next

couple years quite likely during Donald

Trump's uh presidency and I think

there's a view that this has always been

something of corporate hype or

speculation and I think one of the

things I saw in the white house when I

was decidedly not in a corporate

position was trend lines that looked

very clear and what we tried to do Under

the president's leadership was get uh

the US government and our society ready

for these systems before we get into

what do would mean to get

ready what does it mean yeah when you

say extraordinarily capable systems

capable of what the sort of canonical

definition of AGI which again is a term

I don't love is a system it'll be good

if every time you say AGI you caveat

that you dislike the it'll sink in right

yeah people really enjoy that I I'm

trying to get it in the training data

asra um uh a definition of AGI is a

system capable of doing almost any

cognitive task a human can do I don't

know that we'll quite see that in the

next uh four years or so but I do think

we'll see something like that where the

breath of the system is remarkable but

also its depth its capacity to go and

and really push in some cases exceed uh

human capabilities kind of regardless of

the cognitive discipline systems that

can replace human beings in cognitively

demanding jobs yeah or key parts of

cognitive demanding jobs yeah I will say

I am also pretty convinced we're on the

cusp of this so I'm not I'm not coming

at this as a

skeptic but I still find it hard to

mentally live in the world of it so do I

so I use deep research recently which is

a new open eye product it's sort of on

their more pricey tiers so most people I

think have not used it but it it it can

build out something it's more like a

scientific analytical brief in in a

matter of minutes and I work with

producers on the show I hire incredibly

talented people to do very demanding

research work and I asked it to do this

report on the tensions between the

madonian Constitutional system and the

sort of Highly polarized nationalized

parties we now have and what it produced

in a matter of minutes was I would at

least say the median of what any of the

teams I've worked with on this could

produce within days I've talked to a

number of people at firms that do high

amounts of coding and they tell me that

you know by the end of the Year by the

end of next year they expect most code

will not be written by human beings I

don't really see how this cannot have

Labor Market impact I think that's right

I'm not a labor market Economist but I

think that the systems uh are

extraordinarily capable in some ways I'm

very fond of the quote from William

Gibson the future is already here it's

just unevenly distributed and I think

unless you are engaging with this

technology you probably don't appreciate

how good it is today and then it's

important to recognize today is the

worst it's going to be it's only going

to get better and I think that is the

dynamic that in the white house we were

tracking and that I think the next White

House and and our country as a whole is

going to have to track and adapt to in

really short order and what's

fascinating to me what I think is in

some sense the intellectual through line

for almost every AI policy we considered

or implemented is that this is the first

revolutionary

technology um that is not funded by the

Department of Defense basically and if

you go back historically last 100 years

or so nukes space early days of the

internet early days of the

microprocessor early days of large scale

Aviation radar GPS the list is very very

long all of that Tech is fundamentally

comes from DOD money but the the central

government role gave the Department of

Defense and the US government an

understanding of the technology that by

default it does not have an AI and also

gave the US government a capacity to

shape where that technology goes that by

default we don't have an AI there are a

lot of arguments in America about AI the

one thing that seems not to get argued

over that seems almost universally

agreed upon and is the dominant in my

view controlling priority and policy is

it we get to AGI a term I've heard you

don't like yeah before China

does why I do think there are um

profound uh economic and Military and

intelligence capabilities that would be

Downstream of getting to AGI or

transformative Ai and I do think it is

fundamental for US National Security

that we continue to lead AI I think the

the quote that certainly I thought about

a fair amount uh was actually from

Kennedy in his famous rice speech in '

62 the the we're going to the Moon

speech we choose to go to the moon in

this decade and do the other things not

because they are easy but because they

are hard everyone remembers it because

he's saying we're going to the moon but

actually at the end of the speech I

think he gives the better line for space

science

like nuclear science and all technology

has no conscience of its own whether it

will become a Force for good or ill

depends on man and only if the United

States occupies a position of

preeminence can we help

decide whether this new ocean will be a

sea of peace or a new terrifying theater

of war and I think that is true uh in AI

that there's a lot of tremendous

uncertainty about this technology I'm

not an AI evangelist I think there's

huge risks for this technology but I do

think there is a um a fundamental role

for the United States uh in in being

able to shape where it goes which is not

to say we don't want to work

internationally which is not to say we

don't want to work with the Chinese uh

it's worth noting that in the

president's executive order on AI

there's a line in there saying we are

willing to work even with our

competitors on AI safety and the like

but it is worth saying that I think

pretty deeply there's a fundamental role

for America here that we cannot abdicate

paint the picture for me you say there'

be great economic National Security

military risks if China got their first

help me help the audience here imagine a

world where China gets there first so I

think let's look at just the narrow case

of of AI for intelligence analysis and

cyber operations this is I think pretty

out in the open that if you had a much

more powerful AI capability that would

probably enable you to do better cyber

operations um on offense and on defense

what is a operations breaking into an

adversaries Network to collect

information which if you're collecting

in a large enough volume AI systems can

help you analyze and we actually did a

whole big thing through DARPA the

defense Advanced research project agency

um called the AI cyber challenge to test

out ai's capabilities to do this that

was focused on defense because we think

AI could represent a fundamental shift

in how we conduct cyber operations on

offense and defense and I would not want

to live in a world in which uh China has

that capability on offense and defense

and cyber uh and the United States do

not and I think that is true in a bunch

of different domains that are core to

National Security competition my sense

already has been that most people most

institutions are pretty hackable to a

capable State actor not everything but a

lot of them and now both state actors

are going to get better at hacking and

uh they're going to have much more

capacity to do it in the sense that it

you know you can have many more AI

hackers than you can human hackers are

we just about to enter into a world

where we are just much more digitally

vulnerable as normal people and I'm not

just talking about people who the states

might want to spy on but you know you

will get versions of these systems that

just all kinds of Bad actors will have

do you worry it's about to get truly

dystopic what we mean canonically when

we speak of hacking is finding

vulnerability in software exploiting

that vulnerability to get elicit access

and I think it is right that more

powerful AI systems will make it easier

to find vulnerabilities and exploit them

and gain access and that will yield an

advantage to the offensive side of the

ball I think it is also the case that

more powerful AI systems on the

defensive side will make it easier to

write more secure secure code in the

first place reduce the number of

vulnerabilities that can be found and to

better detect the hackers that are

coming in we tried as much as possible

to shift the balance towards the

defensive side of this but I think it is

right that in the the coming years here

the sort of transition period we've been

talking about that there will be a

period in which sort of older Legacy

systems that don't have the advantage of

the newest AI defensive techniques or

software development techniques will on

balance be more vulnerable to a more

capable offensive actor the flip of that

is the question which I know a lot of

people worry about which is the security

of the AI Labs themselves yeah it is

very very very valuable for another

state to get the latest open AI

system and you know the people at these

companies and I've talked to them about

this on the one hand know this is a

problem and on the other hand it's

really annoying to work in a truly

secure way I've worked in the SK for the

last four years a secure room where you

you can't bring your phone and all of

that that that is annoying there's no

doubt about it I yeah how do you feel

about the vulnerability right now of AI

Labs yeah I worry about it and I think

there a hacking risk here I also you

know if you hang out on the right right

San Francisco house party they're not

sharing the model but they are talking

to some degree about the techniques they

use and the like which have tremendous

value I do think the case to come back

to this kind of intellectual through

line of this is National Security

relevant technology maybe world changing

technology that's not coming from the

offices of the government and doesn't

have the kind of government imperat of

security requirements and that shows up

in this way as well we uh in the

National secur memorandum the

president's side tried to Signal this to

the labs and tried to say to them we as

US Government want to help you in this

Mission this was signed in October of

2024 so there wasn't a ton of time for

us to to to build on that but um I think

it's a priority for the Trump

Administration and I can't imagine

anything that is more nonpartisan than

protecting American companies are

inventing the future there's a dimension

to this that I find people bring up to

me a lot is

interesting is that that processing of

information so compared to you know spy

games between the Soviet Union and the

United States we all just have a lot

more data now we have all the satellite

data we I mean obviously will not Eaves

drop on each other but obviously we

Eaves drop on each other and have all

these kinds of things coming in and I'm

told by people who know this better than

I do that there's just a huge choke

point of human beings and their you know

currently fairly rudimentary programs

analyzing that data and that there's a

view that what it would mean to have

these truly intelligent systems that are

able to inhale that and do pattern

recognition is a much more significant

change in the balance of power than

people outside this understand yeah I

think we were pretty public about this

and the president signed a national

security memorandum uh which is

basically the National Security

equivalent of an executive order that

says this is a fundamental area of

importance for the United States I don't

even know the amount of satellite images

that the United States collects every

single day but it's a huge amount and we

have been public about the fact that we

simply do not have enough humans to go

through all of this satellite imagery

and it would be a terrible job if we did

and there is a role for AI uh in going

through uh these images of of hotpots

around the world of Shipping Lines and

all that and analyzing them in an

automated way and surfacing the most

interesting and important ones for human

review and I think at one level you can

look at this and say uh well it doesn't

software just do that and I think that

that some level of course is true at

another level you could say the more

capable that software the more capable

the automation of that analysis the more

intelligent Advantage you extract from

that data and that ultimately leads to a

better for the United States I think the

first and second order consequences of

that are also

striking one thing it implies is that in

a world where you have strong AI the

incentive for spying goes up because if

right now we are choked at the point of

we are collecting more data than we can

analyze well then each marginal piece of

data we're collecting isn't that

valuable I think that's basically true I

think there's two counterveiling aspects

to it the first is uh you you need to

have it I firmly believe you need to

have rights and protections that that

hopefully are pushing back and saying no

there's there's key kinds of data here

including data on your own citizens that

and in some cases citizens of Allied

Nations that you should not collect even

if there's an incentive to collect it

and for all of the flaws of the United

States intelligence oversight process

and all the debates we could have about

this that I think is fundamentally more

important for the reason you suggest in

the era of of tremendous AI systems how

frightened are you by the National

Security implications of all this

which is to say that the possibilities

for surveillance States so Sam Hammond

who's a economist at the foundation for

American innovation he had this piece

called 95 thesis on AI and one of them

that I think about a lot is he makes

this point that a lot of laws right

now if we had the capacity for perfect

enforcement would be constricting like

extraordinarily constricting right laws

are written knowing that

human labor is scarce and you know and

and there's this question of what

happens when the surveillance state gets

really good right what happens when AI

makes the police state a very different

kind of thing than it is now you know

what happens when we have like you know

Warfare of endless drones right I mean

the company Ander has become like a big

you know you hear about them a lot now

they have a they have a relationship I

believe with open AI um uh Palante in a

relationship with anthropic right we're

about to see a real change in this in a

way that I think is from the National

Security side frightening and and there

I very much get why we don't want China

way ahead of us like I get that entirely

but just in terms of the capacities it

gives our own

government how do you think about that I

would decompose essentially this

question about Ai and autocracy or the

surveillance however you want to Define

into two parts the first is the China

piece of this how does this play out in

a state that is truly in its bones in

autocracy and and doesn't even make any

pretense

towards democracy and the like and I

think we could probably agree pretty

quickly here this this makes very

tangible of something that you know is

probably core to the aspirations of

their society of like a level of control

that only an AI system could help uh

bring about that I just find terrifying

as an aside I think there's there's a

saying in both Russian and Chinese

Something Like Heaven is high and the

emperor is far away which is like

historically even in those autocracies

there was some kind of space where the

the state couldn't intrude um because of

the scale and the the breath of the the

nation

and it is the case that in those

autocracies I think AI would would make

the force of government power worse then

there's a more interesting question in

the United States basically War

relationship between Ai and and

democracy and I think I share some of

the the discomfort here there have been

thinkers historically who who have said

you know part of the ways in which we

revise our laws are people break the

laws and and there's a space for that

and I think um there is a a humanness to

our justice system that uh I wouldn't

want to lose and to the enforcement of

justice that I wouldn't want to lose and

we task the Department of Justice and uh

running a process and thinking about

this and coming up with principles for

the use of AI in criminal justice I

think there's in some cases advantages

to it like cases are treated alike uh

with the with the machine but also I

think there's tremendous risk of bias

and discrimination and so forth U

because the systems are flawed uh and in

some cases because the systems are uh

ubiquitous and I I do think there is a

risk of a fundamental encroachment on

rights from the widespread unche use of

AI uh in the law enforcement system that

we should be very alert to and that I as

a citizen um have grave concerns about I

find this all makes me incredibly

uncomfortable and one of the reasons is

that there is

a what's WR to put

this it's like we are summoning an ally

right we are trying to build an alliance

with another uh like an almost

interplanetary Ally and we like we are

in a competition with China to make that

Alliance but we don't understand the

Ally and we don't understand what it

will mean to let that Ally into all of

our systems and all of our planning as

best I understand it every company

really working on this every government

really working on this believes that in

the not too distant future you're going

to have much better and faster and

dominant decision-making Loops by being

able to make much more of this

autonomous to to the AI right once you

get to the what we're talking about as

AGI you want to turn over a fair amount

of your decision-making to it so we are

rushing towards that because we don't

want the other guys to get their first

without really

understanding what that is or what that

means it seems like a like a potentially

historically dangerous thing that AI

reached maturation at the exact moment

that that the US and China are in this

like luidi trap style race for

superpower dominance that's a pretty

dangerous set of incentives in which to

be creating the next turn in

Intelligence on this planet yeah there's

a lot to unpack here so just go in order

but basically bottom line I think I in

the white house and now post White House

greatly share a lot of this discomfort

and I think part of the appeal for

something like the export controls is it

identifies a choke point that can

differentially slow the Chinese down

create space for the United States to

have a lead ideally in my view to spend

that lead on safety and coordination and

not rushing ahead um including again

potentially coordination with the

Chinese while not exacerbating this arms

race Dynamic I would not say that we

tried to race ahead in applications to

National Security so part of the

National Security memorandum is a pretty

lengthy kind of description of what

we're not going to do with AI systems

and a whole list of prohibited use cases

and then high impact use cases and

there's a governance and risk you're not

in power anymore well that's a fair

question now they haven't repealed this

the Trump Administration has not

repealed this but I do think it's fair

to say that um for the period while we

had power the foundation we were trying

to build with AI we were trying we were

very conent to the dynamic you were

talking about a race to the bottom on

safety and we were trying to to guard

against it even as as we try to assure a

position of us preeminence is there

anything to the the the concern that by

treating China as such a such an

antagonistic competitor on this who we

will do everything including export

controls on Advanced Technologies to

hold them back that we have made them

into a more intense competitor I mean

there is

a I do not want to be naive about the

Chinese system or the ideology of the

CCP like they want strength and

dominance and to see the next era be a

Chinese era so maybe there's nothing you

can do about this but it is pretty damn

antagonistic to try to choke off the

chips for the central technology of the

next era to the other biggest country I

don't know that it's pretty pretty

antagonistic to say we are not going to

sell you the most advanced technology in

the world that does not in itself that's

not a declaration of war um that is not

even self a declaration of a Cold War I

I think it is just saying this

technology is incredibly important do

you think that's how they understood it

this is more academic than you want but

my my uh you know academic research when

I started as a professor was basically

on the the the cities trap or what in

Academia would call a security dilemma

of how Nations misunderstand each other

so I'm sure the Chinese and United

States misunderstand each other um at

some level in this area but I

think the plain reading of the facts is

that not selling chips to them I don't

think is a declaration but I don't think

they do misunderstand us I mean maybe

maybe they see it differently but I I

think you're being a little look I'm

aware of how politics in Washington

works I've talked to many people during

this I've seen the turn towards a much

more confrontational posture with China

I know that Jake Sullivan and and

President Biden you know wanted to call

this strategic competition and not a new

Cold War and and I get all that I think

it's true and also we have just talked

about and you did not argue the point

that our dominant view is we need to get

to this technology before they do I

don't think they look at this like oh

you know like nobody would ever sell us

the top technology I think they

understand what we're doing here to some

degree I don't want to trigger with this

I'm sure they do see it that way on the

other hand we set up a AI dialogue with

them and you know I flew Geneva and met

them and and we tried to talk to them

about AI safety and the like so I do

think uh in a area as complex as AI you

can have multiple things be true at the

same time I don't regret for a second uh

the export controls and I think frankly

we we are proud to have done them when

we did them because it has helped ensure

that here we are a couple years later we

retain the edge in AI for as good as and

talented as deep seek is what made deep

seek such a a shock I think to the

American system was here's a system that

appeared to be trained on much less

compute for much less money that was

competitive at a high level with our

Frontier

systems how did you understand what deep

seek

was and what assumptions it required

that we rethink or don't yeah let's just

take one step back so we're tracking the

history of deep seek here so we'd been

watching deep seek in the white house

since November of 23 or thereabouts when

they put out their first coding system

um and there's no doubt that deep seek

Engineers are extremely talented and

they got better and better of their

systems throughout 2024 we were hardened

when their CEO said I think the biggest

impediment to what deep seek was doing

was not their inability to get money or

talent but their inability to get

Advanced chips clearly they still did

get some chips that they some they

bought legally some they smuggled uh so

it seems and then in December of 24 they

came out with a system called version

three deep seek version three which

actually I think is one that should have

gotten the attention um it didn't get a

ton of attention but it did show they

were making strong algorithmic progress

in basically making systems more

efficient and then in January of 25 they

came out with a system called R1 r1's

actually not that unusual no one expect

that to take a lot of computing power

just is a reasoning system that um

extends the underlying um V3 system

that's a lot of nerd speak the key thing

here is when you look at what deeps has

done I don't think the media hype around

it was warrant

and I don't think it changes the

fundamental analysis of of what we are

doing they still are constrained by

computing power we should tighten the

screws and continue to constrain them

they're smart their algorithms are

getting better but so are the algorithms

of us companies and uh this I think

should be a reminder that uh the ship

controls are important China is a worthy

competitor here and we shouldn't take

anything for granted but I don't think

this is the a time to say the sky is

falling or the fundamental scaling laws

have broken where do you think they got

their performance increases from they

have smart people there's no doubt about

that we read their papers they're

they're smart people who are doing

exactly the same kind of algorithmic

efficiency work that companies like

Google anthropic and open AI are doing

one common argument I heard on the left

Lena Khan made this this point actually

in in our in our Pages was that this

proved our whole Paradigm of AI

development was wrong that we were

seeing we did not need all this compute

we were seeing we did not need these

these giant mega companies that this was

showing a way towards like a

decentralized almost solar Punk version

of of AI development

and that in a sense the American system

and and Imagination been captured by

like these three big companies but what

we're seeing from China was that that

wasn't necessarily needed we could do

this on less energy fewer chips less

footprint do you buy that I think two

things are true here the first is there

will always be a frontier or at least

for the foreseeable future they'll be a

frontier that is computationally and

energy um intensive and our company we

want to be at that Frontier those

companies have very strong incentive to

look for efficiencies and they all do

they all want to get every single L

juice of insight from each squeeze of

computation they will continue to need

to push the frontier and I don't think

there's a free lunch waiting in terms of

they're not going to need more computing

power and more energy for the next

couple years and then in addition to

that there'll be kind of slower

diffusion that lags the frontier where

algorithms get more efficient fewer

computer chips are required less energ

is required and that we need as America

to to win both those competitions one

thing that you see around the export

controls the AI firms want the export

controls when deep seek rocked the US

Stock Market it rocked it by making

people question nvidia's long-term worth

and Nvidia very much doesn't want these

export controls so you at the White

House where I'm sure at the center of a

bunch of this lobbying back and forth

how do you think about this every AI

chip every Advanced AI chip that gets

made will get sold the market for these

chips is extraordinary right now I think

for the fore future so I think our view

was uh we put the export controls on but

Nvidia didn't think that the stock

market didn't think that we put the

export controls on the first ones in

October 2022 Nvidia stock has 10 xed

since then I'm not saying we should do

the export controls but I want you to

take the strong version of the argument

not the weak one I don't think nvidia's

CEO is wrong that if we say Nvidia

cannot export its top chips to China

that that in some mechanical way in the

long run reduces the market for

invidious chips sure I think I think the

dynamic is right I'm not suggesting

they're you know if they had a bigger

Market they could charge on the margins

more that's obviously the supply and

demand here I think our analysis was um

considering the importance of these

chips and the AI systems they make to US

National Security this is a trade-off

that's worth it and Nvidia again has

done very well since we put the export

controls out and I agree with that the B

Administration was also generally

concerned with AI safety I think it was

influenced by people who care about AI

safety and that's created a kind of

backlash from the

accelerationist or what gets called the

accelerationist side of this debate so I

want to play a clip for you from Mark

andreon who is obviously very

significant ventor capitalist a top

Trump adviser uh describing the

conversations he had with the Biden

Administration on AI and and how they

sort of radicalized him in the other

direction Ben and I went to Washington

in May of 24 and you know we couldn't

meet with Biden because as it turns out

at the time nobody could meet with Biden

uh but we were able to meet with senior

staff and so we we met with very senior

people in the White House you know in

the in the inter core um and we

basically relate our concerns about Ai

and their response to us was yes the

national agenda on AI as we will

Implement in the B Administration and in

the second term is we are going to make

sure that AI is going to be only a

function of two or three large companies

we will directly regulate and control

those companies um there will be no

startups this whole thing where you guys

think you can just like start companies

and write code and release code of the

internet like those days are over that

not happening the conversation he's

describing there was that were you part

of that conversation I met with him once

I I don't know exactly but we I met with

him once would that characterize the

conversation he had with you he talked

about concerns related to startups and

competitiveness and the like my view on

this as you look at our record on

competitiveness it's pretty clear that

we want a dynamic ecosystem so the AI

executive order which president Trump

just repealed had a pretty lengthy

section on competitiveness the Office of

Management and budget management memo

which governs how the US government buys

AI had a whole carve out in it or a call

out in it saying we want to buy from a

wide variety of vendors the chips and

science act has a bunch of things in

there about competition so I think our

view on competition is pretty clear now

I do think there are structural Dynamics

related to scaling laws on the like that

will force things towards uh big

companies that I think in any respects

we were we were pushing against uh and I

think I think the track record is pretty

clear of us and petition I think the

view that I understand him as arguing

with which is a view I've heard from

people in the safety Community but not a

view I ne heard from the B

Administration was it you will need to

regulate the frontier models of the

biggest labs when it gets efficiently

powerful and in order to do that you

will need there to be controls on those

models you just can't have the model

weights and everything floating around

so everybody can run this on you know

their home laptop I think that's the T

he's getting it it gets it a bigger

attention we'll talk about in a minute

but which is how much to regulate this

incredibly powerful and fast changing

technology such that on the one hand

you're keeping it safe but on the other

hand you're not overly slowing it down

or making it impossible for smaller

companies to comply with these new

regulations as they're using more and

more powerful systems yeah so in the

president's executive order we actually

tried to wrestle with this question and

we didn't have an answer when that order

was signed in October of 23 and what we

did on the open source question in

particular and I think we should just be

precise here at the risk of being

academic again what we're talking about

open Weight Systems can you just say

what what weights are in this context

and then what open weights are yeah so

when you have the training process for

an AI system you run this uh algorithm

through this huge amount of

computational power that processes the

data the output at the end of that

training process Loosely speaking and I

stress this is the loosest possible

analogy they are roughly akin to the the

strength of connections between the

neurons and your brain and in some sense

you could think of this as the the raw

AI system and when you have these

weights one thing that some companies

like meta and deep seek choose to do is

they publish them out on the internet

which makes them we call them openweight

systems I'm a huge believer in the open

source ecosystem many of the companies

that publish the weights for their

system do not make them open source they

don't publish the code and the like so I

don't think they should get the credit

of being called open source systems at

the risk of being pedantic but open

Weight Systems is something we thought a

lot about in 23 and 24 and we sent out a

a pretty wide ranging um request for

comment from a lot of FK for a lot of

folks we got a lot of um comments back

and what we came to in the report that

was published in July or so of 24 was

there is not evidence yet to constrain

the openweight ecosystem that the

openweight ecosystem does a lot for

Innovation and the like which I think is

manifestly true but that we should

continue to monitor in this as the

technology gets better basically exactly

in the way that you described so we're

talking here a bit about the the sort of

race Dynamic and and the safety Dynamic

when you were getting those comments not

just on the open weight models but also

when you were talking to the heads of

these labs and people were coming to you

what did they want what would you say

was like the consensus to the extent

there was one from AI world of what they

needed to get there quickly and also

because I know that many people in these

labs are worried about what it would

mean if these systems were UNS safe what

was what you would describe as a ensus

on safety I mentioned before this this

this core intellectual Insight of this

technology for the first time maybe in a

long time is a revolutionary one not

funded by the government and its early

incubator

days that was the theme from the labs

which is was sort of a like don't you

know we're inventing something very very

powerful ultimately it's going to have

implications for the kind of work you do

in National Security the way we organize

our society and more than any kind of

individual policy

request they were basically saying like

get ready for this the one thing that we

did that could be the closest thing we

did to any kind of Regulation there's

one action which was after the labs made

voluntary commitments to do safety

testing we said you have to share the

safety test results with us and you have

to help us understand where the

technology is going and that only

applied really to the top couple Labs

the labs never knew that was coming

weren't all thrilled about it it when it

came out so the notion this was kind of

a a regulatory capture that we were

asked to do this is simply not true but

I in my experience never got you know

discret individual policy lobbying from

the laps I got much more this is coming

it's coming much sooner than you think

make sure you're ready to the degree

that they were asking for something in

particular it was maybe a corollary of

that of we're going to need a lot of

energy and we want to do that here in

the United States and it's really hard

to get the power here in the United

States but that is has become a pretty

big question if this is all as potent as

we think it will be and you end up

having a bunch of the data centers

containing all the model weights and and

and everything else in a bunch of uh

like Middle Eastern Pro States because

they hpthe speaking hypothetically

because they will give you huge amounts

of energy access in return for just at

least having some purchase on this AI

World which they don't have the internal

engineering talent to be competitive in

but maybe can get some of it located

there and then there's some technology

ex right like there is something to this

question yeah and and this is actually I

think an area of of bipartisan agreement

which we can get to but this is

something that we really started to pay

a lot of attention to in 20 later part

of 23 and most of 24 when it was clear

this was going to be a ball neck and in

the last week or so in office President

Biden signed a AI infrastructure

executive order which has not been

repealed which basically tries to

accelerate the power development and the

perming of power and data centers here

in the United States basically for the

reason that you mentioned now as someone

who truly believes in climate change and

environmentalism and clean power I

thought there was a double benefit to

this which is that if we did it here in

the United States it could catalyze the

clean energy transition and like and

these companies for a variety of reasons

in general are willing to to pay more

for clean energy and on things like

geothermal and the like our Hope was we

could catalyze that development and bend

the cost curve and have companies be the

early adopters of that technology so

we'd see a win on the climate side as

well so I I would say there is a there

are Waring cultures around how to

prepare for for AI and I sort of

mentioned AI safety and and Ai

accelerationism and GD Vance just went

to the sort of big AI Summit in Paris

and I'll play a clip of what he

said I'm not here this morning uh to

talk about AI safety which was the title

of the conference a couple of years ago

I'm here to talk about AI opportunity

when conferences like this convene to

discuss a cuttingedge technology often

times I think our response is to be too

self-conscious too risk averse but never

have I encountered a breakthrough in

Tech that so clearly calls us to do

precisely the opposite now our

Administration the Trump Administration

believes that AI will have countless

revolutionary applications and economic

in Innovation job creation National

Security Health Care Free expression and

Beyond

and to restrict its development now

would not only unfairly benefit

incumbents in the space it would mean

paralyzing one of the most promising

Technologies we have seen in Generations

what do you make of

that so I think he is setting up a

dichotomy there that I don't quite agree

with and the irony of that is if you

look at the rest of his speech which I

did watch there's actually a lot that I

do agree with so he talks for example I

think he's got four pillars in the

speech one's about centering the

importance of workers ones about

American uh preeminence and like those

are entirely consistent with the actions

that we took and the philosophy that I

think the administration uh of which I

was a part espoused uh and and that I

certainly believe in so far as what he

is saying is that safety and opportunity

are in fundamental tension then I

disagree and I think if you look at the

history of technology and Technology

adaptation the evidence is pretty clear

that the right amount of safety um

action unleashes opportunity and in fact

unleashes speed so one of the examples

that uh we studied a lot and talked to

the president about was the early days

of railroads and in the early days of

railroads there were tons of accidents

and crashes and deaths and people were

not inclined to use railroads as a

result and then what started happening

was safety standards and safety

technology um block signaling so that

trains could know when they were in the

same area uh air brakes so that trains

could break more efficiently uh

standardization of train track widths

and gauges and the like and this was not

always popular at the time but with the

benefit of hindsight it is very clear

that that kind of uh technology and to

some degre policy development of safety

standards made the American railroad

system in the late 1800s and I think

this is a pattern that shows up a bunch

throughout the history of technology to

be very clear it is not the case that

every safety regulation every technology

is good and there certainly are cases

where you can overreach and you can slow

things down and choke things off but I

don't think it's true that there's a

fundamental tension between safety and

opportunity it's interesting because I I

don't know how to get this point of

Regulation right I think the

counterargument to to uh vice president

Vance is

nuclear so nuclear power is a technology

that both held extraordinary promise

maybe still does and also you could

really imagine every country wanting to

be in the lead on but the series of

accidents which most of them did not

even have a particularly significant uh

body count y were so frightening to

people that the technology got regulated

to the point that certainly all of

nuclear's Advocates believe it has been

largely strangled in the crib from what

it could be the question then is when

you look at the actions we have taken on

AI are we strangling in the crib and

have we taken actions that are akin to

I'm not saying that we've already done

it I'm saying that look if these systems

are going to get more powerful and

they're going to be in charge of more

things things are both going to go wrong

and they're going to go weird it's not

possible for it to be otherwise right to

roll out some something this new in a

system as complex as human society and

so I think there's going to be this

question of what are the regimes that

make people feel comfortable moving

forward from those kinds of moments yeah

I think that's a profound question I

think what we tried to do in the Biden

Administration was set up the kind of

institutions in the government to do

that in as cleare eyed tech-savvy way as

possible again with the one exception of

the safety test result sharing which

some of the CEOs estimate cost them one

day of employee work we did not put any

anything close to regulation in place we

created something called the AI safety

Institute purely National Security

focused cyber risk bio risks AI accident

risks uh purely voluntary and that has

relationships memor of understanding

with anthropic with open AI even with

xai Elon company and basically I think

we saw that as an opportunity to bring

AI expertise into the government to

build relationship between public and

private sector in a voluntary way and

then as the technology develops it will

be up to

now the Trump Administration decide what

they want to do with it I think you are

quite diplomatically understating though

what's a a genuine disagreement here and

what I would say Vance speech was

signaling was the arrival of a different

culture in the government around AI

there's been an AI safety culture where

and he's making this point explicitly

that we have all these conferences about

what could go wrong and he is saying

stop it yes maybe things could go wrong

but instead we should be focused on what

could go right and and I would say

frankly this this is like the Trump musk

which I think is some ways the right way

to think about the administration their

generalized view if something goes wrong

we'll deal with a thing that went wrong

afterwards right but what you don't want

to do is move too slowly because you're

worried about things going wrong better

to break things and fix them then have

moved too slowly in order not to break

them I think it's fair to say that there

is a a cultural difference between the

Trump Administration and us on on some

of these things and but I I also you

know we held conferences on what you

could with AI and the benefits of AI we

talked all the time about how you uh

need to mitigate these risks but you're

doing so so you can capture the benefits

and I'm someone who you know reads an

essay uh like Dar Amad SE of anthropics

machines of love and grace about the

upside of AI and says there's a lot in

here we can agree with and president's

executive order said we should be using

AI more um in the executive branch so I

I I I hear you on the cultural

difference I get that but I think when

the rubber meets the road um we were

comt with the notion that you could both

realize the opportunity of AI while

doing it safely and now that they are in

power they will have to decide how do

they translate vice president Vance's

rhetoric into a governing policy and and

my understanding of their executive

order is they've given themselves six

months to figure out what they're going

to do and I think we should judge them

in what they do let me ask about the

other side of this because what I liked

about Vance's speech is I think he's

right that we don't talk enough about

opportunities but more than that we are

not preparing for opportunities so if

you imagine that will have the effects

and possibilities that its backers and

and Advocates hope one thing that that

implies is that we are going to start

having a much faster pace of the

discovery or proposal of Novel drug

molecules a very high promise the idea

here from people I've spoken to is that

a should be able to ingest an amount of

information and build sort of modeling

of diseases in the the human body that

that could get us a much much much

better drug Discovery pipeline

if that were true then you can ask this

question well what's the choke point

going to be and our drug testing

pipeline is incredibly cumbersome it's

very hard to get the animals you need

for trials very hard to get the human

beings you need for trials right you

could do a lot to make that faster to to

prepare it for a lot more coming in and

this is true in a lot of different

domains right education Etc I think it's

pretty clear that the choke points will

become the difficulty of doing things in

the real world and I don't see Society

also preparing for that right we're not

doing that much on the safety side maybe

because we don't know what we should do

but also on the opportunity side you

know this question of how could you

actually make it possible to translate

the benefits of the stuff very fast

seems like a much richer conversation

I've seen anybody seriously having yeah

I I think I I basically agree with all

of that I think the conversation when we

were in the government especially in 23

and 24 uh was starting to happen we

looked at the the clinical trials saying

I you've read about healthcare for

however long I don't claim expertise on

Healthcare but it does seem to me that

we want to get to a world where we can

take the the breakthroughs including

breakthroughs from AI systems and

translate them to Market much faster

this is not a hypothetical thing it's

worth noting I think quite recently

Google came out with I think they call

the co- scientist Nvidia and the arc

Institute which does great work um had

the most impressive biod design model

ever that that has a much more detailed

understanding of uh biological molecules

a group called future house has done

similarly great work in science so I

don't think this is a hypothetical I

think this is happening right now and I

agree with you that there's a lot that

can be done institutionally and

organizationally to get the federal

government ready for this I've been

wandering around Washington DC this week

and talking to a lot of people involved

in different ways in the Trump

Administration or advising the Trump

Administration different people from

different factions of you know what what

I think is the modern right I've been

surprised how many people

understand either what Trump and musk

and Doge are doing or at least what it

will end up allowing as related to AI

including people I would not really

expect to hear that from not Tech right

people but what they basically say is

there is no way in which the federal

government as constituted six months ago

moves at the speed needed to take

advantage of this technology either to

integrate it into the way the government

works or for the government to take

advantage of what it can do that we are

too cumbersome to endless inter agency

processes too many rules too many

regulations you have to go through too

many people that if the whole point of

AI is that it is this unfathomable

acceleration of cognitive work the

government needs to be stripped down and

rebuilt to take advantage of it and like

them hate them what they're doing is

stripping the government down and

rebuilding it and maybe they don't even

know what they're doing it for but one

thing it will allow is a kind of

creative destruction that you can then

begin to insert AI into at a more ground

level do buy that it feels kind of

orthal from what I've observed from Doge

I mean I I think Elon is someone who

does understand what AI can do but I

don't know how starting with usaid for

example prepares the US government to

make better AI policy so I guess I don't

buy it that that is the the motivation

for doge is there something to the

broader argument and and I will say I do

buy not the argument about Doge which

sort of make the same point you just

made what I do buy is that I know how

the federal government works pretty well

and it is too slow to modernize

technology it is too slow to work across

agencies it is too slow to radically

change the way things are done and take

advantage of things it could be

productivity

enhancing I I couldn't agree more I mean

the existence of my job in the White

House the White House special advisor

for AI which David sax now is and I uh

had this job in 2023 existed because

President Biden said very clearly

publicly and privately we cannot move at

the typical government Pace we have to

move faster here I think we probably

need to be careful and you know I'm not

here for stripping it all down but I

agree with you we have to move much

faster so another major part of Vice

President Vance's speech was signaling

to the

Europeans that we are not going to sign

on to complex multilateral negotiations

and regulations that could slow us down

and that if they passed such regulations

anyway in a way that we believe was

penalizing our AI companies we would

retaliate how do you think about the

differing position the new

Administration is moving into VV Europe

and its approach its broad approach to

Tech

regulation yeah I think the honest

answer here is we had conversations with

Europe as they were drafting the EU AI

act but at the time that I was in the

eui Act was was still kind of nent and

the ACT had passed but a lot of the

actual details of it had been kicked to

a process that my sense is still

unfolding So speaking of slow moving I

bureaucracies uh exactly exactly so

maybe this is a failing on my part I did

not have particularly detailed

conversations with the Europeans Beyond

a general kind of articulation of our

views they were respectful uh we were

respectful but I think it's fair to say

we were taking a different approach than

they were taking and uh we were probably

in so far as safety and opportunity are

a dichotomy which I I don't think they

are a pure dichotomy uh we were were we

were ready to move very fast in the

development of AI one of the other

things that Vance talked about and that

you said you agreed with is making AI

pro-worker what does that mean it's a

it's a vital question I think we

instantiated that in a couple of

different principles the first is that

AI in the workplace uh needs to be

implemented in a way that is respectful

of workers and the like and I think um

one of the things I know the president

thought a lot about was you it is

possible for AI to make workplaces worse

and in a way that is dehumanizing and

degrading and ultimately destructive for

workers so that is sort of a first

distinct piece of it that I don't want

to neglect the second is I think we want

to uh have ai deployed across our

economy in a way that increases workers

agencies and capabilities and I think we

should be honest that there's going to

be a lot of transition in the economy as

a result of AI you can find Nobel Prize

wi economists who will say it won't be

much you can find other folks who will

say it'll be a ton I tend to lead

towards the it's going to be a lot side

but I'm not a labor Economist and uh the

line that that vice president Vance used

is the exact same phrase that President

Biden used which is give workers a seat

at the table um in that transition and I

think that is a fundamental part of what

we're trying to do here and I presume

what they're trying to do here so I've

sort of heard you beg off on this

question a little bit by saying you're

not a labor Economist uh I will say the

I'm not a labor Economist you're not I

will promise you the labor Economist do

not know what to do about AI yeah you

were the top adviser for AI yeah you

were at the nerve center of the

government's information about what is

coming

if this is half as big as you seem to

think it is it's going to be the single

most disruptive thing to hit labor

markets ever given how compressed the

time period in which it will arrive is

right it took a long time to lay down

electricity it took a long time to build

railroads I I think that is basically

true but I want to push back a little

bit so I do think we are going to see a

dynamic in which

it will hit parts of the economy first

it will hit certain firms first but it

will be an uneven distribution acoss I

think it will be uneven and that's I

think what will be destabilizing about

it in part right if it were just even

then you might just come up with an even

policy to do something about it sure but

precisely because it's not even and it's

not going to put I don't think 42% of

the labor force out of work overnight

no let me give you an example the kind

of thing I'm worried about and I've

heard other people worry about there are

a lot of

19-year-olds in college right now

studying

marketing there are a lot of marketing

jobs that AI frankly can do perfectly

well right now as we get better at

knowing how to direct I mean one of the

things exist slow this down is simply

firm adaptation yes but the thing that

will happen very quickly is you have

firms that are built around AI right

it's going to be harder for the big

firms to integrate it but what you're

going to have is new entrance who are

built from the ground up with their

their their organization is built around

you know one person overseeing these

like you know seven systems and so you

might just begin to see Triple the

unemployment among marketing

graduates I'm not convinced you'll see

that in software Engineers because I

think AI is going to both you know take

a lot of those jobs it also create a lot

of those jobs because there's going to

be so much more demand for software but

you could see it happening somewhere

there there's just a lot of jobs that

are doing work behind a

computer and as companies

absorb machines that can do work by the

computer for you that will change their

hiring you must have heard somebody

think about this you guys must have

talked about this we did talk to

economists and and try to texture this

debate uh in 23 and 24 I think the trend

line is even clearer now than it was

then I think we knew this was not going

to be a 23 and 24 question frankly to do

anything robust about this is going to

require Congress and was just not in the

cards at all so it was more of a an

intellectual exercise than it was a

policy polies begin as intellectual

exercises yeah I think I think that's

fair um I think the advantage to AI that

is in some ways a countervailing force

here is that it will increase the amount

of agency for individual people so I do

think we will be in a world in which the

19-year-old or the 25-year-old will be

able to use a system to do things they

were not able to do before and I think

in so far as the thesis we're batting

around here is that intelligence will

become a little bit more commoditized

what will stand out more in that world

is agency and the capacity to do things

or initiative and the like and I think

that could in the aggregate lead to a

pretty Dynamic economy and the economy

you're talking about of small firms and

uh Dynamic ecosystem and robust

competition I think unbalanced at an

economy scale is not in itself a bad

thing I think where I imagine you and I

agree and and maybe vice president Vance

as well agree is we need to make sure

that for individual workers and classes

of workers they're protected in that in

that transition I think we should be

honest that's going to be very hard um

we have never done that well I I

couldn't agree with you more like in a

big way Donald Trump is president today

because we did a shitty job on this with

China this is a kind of like the reason

I'm pushing on this is that we have been

talking about this seeing this coming

for a while and I will say that as I

look around I do not see a lot of useful

thinking here and I grant that we don't

know the shape of it at the very least I

would like to see some IDE is on the

shelf for if the disruptions are severe

what we should think about doing we are

so addicted in this country to an

economically useful tale that our

success is in our own hands it makes it

very hard for us to react with either

Compassion or

realism when workers are displaced for

reasons that are not in their own hands

because of global recessions or

depressions because of globalization

there are always some people with like

the agency the creat creativity the and

they become hyper productive and you

know look at them why aren't you them

but definitely I know you're not saying

that but it's very hard that's such an

ingrained American way of looking at the

economy that we have a lot of trouble

doing you know always should do some

retraining right is are all these people

going to become

nurses right I mean there are things

yeah I can't do like how many plumbers

do we need I mean more than we have

actually but does everybody move into

the trades what were the intellectual

thought exercises that all these smart

people at the White House believe this

was coming you know what were you saying

so I I think yes we were thinking about

this question I think we knew it was not

going to be a question we were going to

confront in the president's term I think

it was we knew it was a question that

you would need Congress for to do

anything about I think I in so far as

what you're expressing here seems to me

to be like a deep dissatisfaction with

the available answers I Shar that I

think a lot of us shared that you know

you can get the the usual stock answers

of a lot of retraining I I share your

sort of doubts that that is the answer

you probably talk to some silicon value

Libertarians or something and they'll

say or or Tech folks and they'll say

well Universal basic income I think I

believe and I think the president

believes there's a kind of dignity uh

that that work brings and and doesn't

have to be paid work but that there

needs to be something that people do

each day that that gives them meaning so

in so far as what you were saying is

like there's a you have a discomfort

with where this is going on the labor

side uh speaking for myself I I share

that I don't know the shape of it I

guess I would say more than that a

discomfort with the quality of thinking

right now sort of across the board but

but I will say on the Democratic side

right because I have you here as a

representative of the past

administration I have a lot of

disagreements with the Trump

Administration to say the least but you

know I do understand the people who say

look Elon Musk David Sachs Mark Andre JD

Vance at the very highest levels of that

Administration or people have spent a

lot of time thinking about Ai and have

like considered very unusual thoughts

about it and I think sometimes Democrats

are a little bit institutionally

constrained for thinking unusually I

take your point on the export controls I

take your point on the exact orders the

the I safety

Institute but to the extent Democrats

are the party want to be imagine

themselves to be the party of the the

working class and to the extent we've

been talking for years about the

possibility of Aid driven

displacements yeah when things happen

you need Congress but you also need

thinking that becomes policies that

Congress do so I guess I'm trying to

like was this not being talked about

there were no meetings there were no you

guys didn't have Claud write up a brief

of of options well you know we

definitely didn't have CLA up a brief

because we had to get over government

use of AI uh see but that's like itself

uh slightly damning yeah I mean I I

think you know Esra I I agree that the

government has to be more forward

leaning on basically all of these

Dimensions it was my job to push the

government to do that and I think on

things like govern use AI we we made

some um progress so I don't think anyone

from the B Administration less of all me

is coming out and saying we solved it I

think what we're saying is like we were

building a foundation for something that

is coming that was not going to arrive

during our time in office um and that

the next team uh is going to as a have

to As a matter of American National

Security and in this case American uh

economic strength and and prosperity uh

address I I will say this gets is

something I find frustrating in the

policy conversation about

AI which is you sit down with somebody

and you start the conversation and like

the most transformative technology

perhaps in human history is landing into

human civilization in a two to three

year time frame and you say wow that

seems like a really big deal what should

we

do and then things get a little hazy

right now maybe we just don't know but

but what I've heard you kind of say a

bunch of times is like look we have done

very little to to hold this technology

back everything is voluntary you know

the only thing we asked was a sharing of

safety data you now income the

accelerationists you know Mark andrewson

has criticized you guys extremely uh

straightforwardly is this policy debate

about anything is it just uh the uh

sentiment of the rhetoric right like if

it's so

big but nobody can quite explain what it

is we need to do or talk about except

for maybe export chip controls like are

we just not thinking creatively enough

is it just not time like match the kind

of calm measure tone of the SE second

half of this with where we started for

me I think there should be an

intellectual humility about before you

take a policy action you have to have

understand some understanding of what it

is you're doing and why so I think it is

entirely intellectually consistent to

look at a transformative technology draw

the lines on the graph and say this is

coming pretty soon without having the

14-point plan of this is what we need to

do in 2027 2028 I think chip controls

are are unique and that this is a

robustly good thing uh that we could do

early to to buy the space I talked about

before but I also think that we tried to

build institutions like the AI safety

Institute that would set the new team up

whether it was us or someone else for

success in managing the technology now

that it's them they will have to decide

as this technology comes on board how do

we want to calibrate this on regulation

what are the kinds of decisions you

think they will have to make in the next

two years you mentioned the open source

one I have a guess where they're going

to land on that but that I think there's

there's an intellectual debate there

that is Rich we resolved it one way by

not doing anything they'll have to

decide do they want to keep doing that

ultimately they'll have to answer a

question of what is the relationship

between the public sector and the

private sector is it the case for

example that the kind of things that are

voluntary now with the AI safety

Institute will someday become mandatory

another key decision is we tried to get

the ball rolling on the use of AI for

National Defense in a way that is

consistent with American values they

will have to decide what does that

continue to look like and do they want

to take some of the safeguards that we

put in place away to go faster so I

think there really is a bunch of

decisions that they are teed up to make

over the next couple years that we can

appreciate they coming on the horizon

without me sitting here and saying I

know with certainty what the answer is

going to be in 2027 and then always our

final question what are three books

you'd recommend to the audience uh one

of the books is the structure of

scientific revolutions uh by Thomas

this is a book that coined the term

Paradigm Shift which basically is what

we've been talking about throughout this

whole conversation of a shift in

technology and scientific understanding

and its implications for society and I

like how uh in this book which was

written in the 1960s gives a series of

historical examples and theoretical

Frameworks for how do you think about a

paradigm shift and then another book

that that has been very valuable for me

is rise of the Machines by Thomas rid uh

and that really tells the story of how

uh machines that were once the play

things of dorks like me became in the

60s and the 70s and the 80s uh things of

National Security importance we talked

about some of the Revolutionary

Technologies here the internet

microprocessors and the like that

emerged out of this intersection between

National Security and Tech development

and I think that history should inform

the work we do today and then the last

book is is definitely an unusual one but

I think is vital and that's a swim in

the pond in the reain by George Saunders

and he's this great essayist and short

story writer and novel writer and he

teaches Russian literature and he in

this book takes uh seven Russian

literature short stories and gives a a

literary interpretation of them and what

strikes me about this book is he's an

incredible writer and this fundamentally

is like the most human endeavor I can

think of he's taking great human short

stories and he's giving a a modern

interpretation of what those stories

mean and I think when we talk about the

kinds of cognitive tasks that are a long

way off for machines I kind of at some

level hope this is one of them that

there's something fundamentally human

that we alone can do uh I'm not sure if

that's true but I hope it's true I'll

say I had him on the show for that book

it's one of my favorite ever episodes

people should check it out Ben be Canan

thank you very much thanks for having me

[Music]