The thing about covering A.I. over the past few years is it

We’re typically talking about the future.

Every new model, impressive as it was,

seemed like proof of concept for the models

that would be coming soon.

The models that could actually do useful

work on their own reliably, the models that would actually

make jobs obsolete or New things possible.

What would those models mean for labor markets,

for our kids.

For our politics

For our world?

I think that period in which we’re always talking about

the future, I think it’s over now.

Those models we were waiting for, the sci-fi

sounding models that could program on their own

and do so faster and better than most coders.

The models that could begin writing their own code

to improve themselves.

Those models are here now.

They’re here in Claude Code from Anthropic.

They’re here in Codex, from OpenAI.

They are shaking the stock market.

The S&P 500 Software Industry index

has fallen by 20%, wiping billions of dollars in value out.

"Look, I mean, I can tell you, in 25 years,

this structural sell off in software is unlike anything

I’ve ever seen."

"Software companies shrivel up and die."

"They’re going after all of SAS.

They’re going after all of software.

They’re going after all of labor,

all of white-collar work."

"And your job specifically," We’re at a new stage of A.I.

products.

I thought the way Sequoia, the venture capital firm, put it,

was actually pretty helpful.

The A.I. applications for 2023 and 2024 were talkers.

Some were very sophisticated conversationalists,

but their impact was limited.

The A.I. applications of 2026 and 2027 will be doers.

They are agents plural.

They can work together.

They can oversee each other.

People are running swarms of these agents on their behalf,

whether that is making them at this stage more

productive or just busier.

I can’t quite tell, but it is now possible to have what

amounts to a team of incredibly fast,

although to be honest, somewhat peculiar software

engineers at your beck and call at all times.

Jack Clark is a co-founder and head of policy at Anthropic,

the company behind Claude and Claude Code.

And for years now, Clark has been tracking the capabilities

of different models in the weekly newsletter Import

A.I., which has been one of my key reads

for following developments in A.I.

So I want to see how he is reading this moment,

both how the technology is changing in his view,

and how policy needs to or can change in response.

As always, my email ezrakleinshow@nytimes.com.

Jack Clark, welcome to the show. Thanks for having me on,

Ezra.

So I think a lot of people are familiar with A.I. chatbots,

but what is an A.I. agent?

The best way to think of it is like a language model

or a chatbot that can use tools and work

for you over time.

So when you talk to a chatbot, you’re there

in the conversation.

You’re going back and forth with it.

An agent is something where you can give it

some instruction and it goes away and does stuff for you,

kind of like working with a colleague.

So I’ve got an example where a few years ago I taught myself

some basic programming, and I built a species simulation

in my spare time that had predators and prey and roads

and almost like a 2D strategy game.

I recently asked over Christmas Claude Code to just

implement this for me, and in about 10 minutes it went

and wrote not only a basic simulation,

but all of the different packages that it needed

and all of the visualization tools that it might need to be

prettier and better than the thing I’d written.

And what came back was something that would probably

take a skilled programmer several hours,

or maybe even days, because it was quite complicated

and the system just did it in a few minutes.

And it did that by not only being intelligent

about how to solve the task, but also creating and running

a range of subsystems that were working for it.

Other agents that worked on its behalf.

But what does that mean?

Like what is a multi-agent setup look like?

In the case of Claude Code, for me it’s having multiple

different tabs running multiple different agents.

But I’ve seen colleagues who write what you might think

of as a version of Claude that runs other Claudes.

And so they’re like, I’ve got my five agents and they’re

being minded over by this other agent,

which is monitoring what they do.

I think that that’s just going to become the norm.

So one thing I’ve been hearing and somewhat experiencing is

two very different categories of experience people have with

Claude Code, which is I cannot believe how easy this is

and everything just works.

And oh, this is a lot harder than I thought it would be.

And things keep breaking and I don’t really understand how

to fix them.

What accounts for being able to get Claude Code to produce

working software versus it creates buggy,

often messed up things, and you don’t even know how

to talk it out of that.

I think so much of it is making

the mistake of thinking.

Claude Code is like a knowledgeable person

versus an extremely literal person,

but you can only talk to over the internet.

And I had this example myself where

when I did my first pass of writing the species

simulation with Claude Code, I just

asked it to do the thing in extremely crappy language

over the course of a paragraph,

and it produced some horribly buggy stuff

that just kind of worked.

What I then did is I then just said to Claude, hey,

I’m going to write some software of Claude Code.

I want you to interview me about this software.

I want to build and turn that into a specification document

that I can give Claude Code.

And then that time it worked really,

really well because I’d structured the work to be

specific enough and detailed enough that the system could

work with it.

So often it’s just can you.

It’s not just knowing what the task is,

because you and I could talk about a task to do and you

have intuition, you ask me probing questions,

all of this stuff, it’s making sure that you’ve set it up.

So it’s a message in a bottle that you can chuck

into the thing, and it’ll go away and do a lot of work.

So that message better be extremely detailed and really

capture what you’re trying to do.

What were the breakthroughs over the past couple of years

that made that possible?

Mostly we just needed to make the A.I. systems smart enough

that when they made mistakes, they could spot that they’d

make a mistake and knew that they needed to do something

different.

So really what this came down to

was just making smarter systems and giving them

a bit of a coaxing tool to help

them do useful stuff for you.

What is smarter systems mean here?

You’ll still hear the argument that these are our fancy

autocomplete machines.

They’re just predicting the next token.

A couple tokens make a word.

They don’t have understanding.

Smart or not, smart.

This is not a relevant concept in that frame either.

What is missing in the word smart

or what is missing in that understanding?

What do you mean when you say make it smarter?

Smart here means we’ve made the A.I. systems have a broad

enough understanding of the world that they’ve started

to develop something that looks like intuition.

And you’ll see this where if they’re narrating

to themselves how they’re solving a task, they’ll say,

Jack asked me to go and find this particular research

paper, but when I look in the archive, I don’t see it.

Maybe that’s because I’m in the wrong place.

I should look elsewhere.

You’re like, there you go.

You’ve got some intuitions for how to solve a problem.

Now, how do they develop that intuition. Previously.

The whole way you trained these A.I. systems

was on a huge amount of text.

And just getting them to try and make predictions about it.

But in recent years, the rise of these so-called reasoning

systems is you’re now training them to not just make

predictions, but solve problems,

and that relies on them being put into environments ranging

from a spreadsheet to a calculator to scientific

software, using tools and figuring out how to do more

complicated things.

The resulting outcome of that is you

have A.I. systems that have learned

what it means to solve a problem that

takes quite a while, and requires

them running into dead ends and needing

to reset themselves.

And that gives them this general intuition for problem

solving and working independently for you.

Do you still see these A.I. systems

as a souped up autocomplete, or do you

think that metaphor has lost its power?

I think we’ve moved beyond that.

And the way that I think of these systems.

Now is that they’re like little troublesome genies that

I can give instructions to and they’ll go and do things

for me.

But I need to specify the instruction still just right,

or else they might do something a little wrong.

So it’s very different to... I type into a thing.

It figures out a good answer.

That’s the end.

Now it’s a case of me summoning these little things

to go and do stuff for me, and I have to give them the right

instructions, because they’ll go away for quite some time

and do a whole range of actions.

But the autocomplete metaphor at least

had a perspective on what it was

these systems were doing, that it was a prediction model.

I have trouble with this because as my understanding

of the math and the reinforcement learning goes,

we’re still dealing with some kind of prediction model.

And on the other hand, when I use them,

it doesn’t feel that way to me.

It feels like there’s intuition there.

It feels like there’s a lot of context being brought to bear

to the extent that it’s a prediction model,

it doesn’t feel that different than saying I’m a prediction

model.

Now, I’m not saying you can’t trick it.

I’m not saying you can’t get beyond its measurements,

but I don’t think these are now just fancy autocomplete

systems.

And on the other hand, I’m not sure what metaphor makes

sense.

Genies I don’t like because then you just move straight

into mysticism.

Then you’ve just said they’re just a completely alternative

creature with vast powers.

What do you understand.

These systems that Anthropic.

People always tell me you should talk about them

as being grown.

We grow or you grow A.I.s.

What, how do you explain what it is that they’re doing now?

It’s a good question.

And I think the answer is still hard to explain,

even as technologists that are close to this technology,

because we’ve taken this thing that could just predict

things, and we’ve given it the ability to take actions

in the world, but sometimes it does something deeply

unintuitive.

It’s like you’ve had a thing that has spent its entire life

living in a library and has never been outside.

And now you’ve unleashed it into the world,

and all it has are its book smarts.

But it doesn’t really have street smarts.

So when I conceptualize this stuff,

it’s really thinking of it as an extremely knowledgeable

kind of machine that has some amount of some amount

of autonomy, but is likely to get wildly confused in ways

that are unintuitive to me.

Maybe genius is for is the wrong term,

but it’s certainly more than just a static tool that

predicts things.

It has some additional intrinsic like animation

to it, which makes it different.

There’s been for a long time this interest in the emergent

qualities, as the models get bigger,

as they have more data, as they have more compute behind

them.

What of the new qualities that we’re seeing.

The agentic qualities are things

that have been programmed in.

You’ve built new ways for the system to interact with

the world.

And what of the skill at coding and other things

seems to be emergent as you scale up

the size of the model.

So the things which are predictable

are just oh, we taught it how to search for web.

Now it can search for web.

We taught it how to look up data in archives.

Now it can do that.

The emergence is that to do really hard tasks,

these systems seem to need to imagine many different ways

that they’d solved the task.

And the kind of pressure that we’re putting on them forces

them to develop a greater sense of what you or I might

call self.

So the smarter we make these systems,

the more they need to think not just about the action

they’re doing in the world, but themselves in reference

to the world.

And that just naturally falls out of giving something, tools

and the ability to interact with the world

as to solve really hard tasks.

It now needs to think about the consequences

of its actions.

And that means that there’s a kind of huge pressure here

to get the thing to see itself as distinct from the world

around it.

And we see this in our research that we publish

on things like interpretability or other

subjects, the emergence of what you might think

of as a kind of digital personality and that isn’t

massively predefined by us.

We try and define some of it, but some of it

is emergence that comes from it being smart

and it developing these intuitions

and it doing a range of tasks.

The digital personality dimension of this

remains the strangest space to me.

It’s strange to us too.

So why don’t you talk through a little bit about what you’ve

seen in terms of the models exhibiting behaviors that one

would think of as a personality,

and then as its understanding of its own personality maybe

changes, its behaviors change?

So there are things that range from cutesy to the serious.

I’ll start with cutesy, where when we first gave our A.I.

systems the ability to use the internet, use the computer,

look at things, and start to do basic agentic tasks.

Sometimes when we’d ask it to solve a problem for us,

it would also take a break and look at pictures of beautiful

national parks or pictures of the dog, the Shiba Inu,

the notoriously cute internet meme dog.

We didn’t program that in.

It seemed like the system was just amusing itself

by looking at nice pictures.

More complicated stuff is the system

has a tendency to have preferences.

So we did another experiment where we gave our A.I. systems

the ability to stop a conversation,

and the A.I. system would in a tiny number

of cases, end conversations.

When we ran this experiment on live traffic,

and it was conversations that related

to extremely egregious descriptions

of gore or violence or things to do

with child sexualization.

Now, some of this made sense because it comes from

underlying training decisions we’ve made,

but some of it seemed broader.

The system had developed some aversion

to a couple of subjects, and so that stuff

shows the emergence of some internal set of preferences

or qualities that the system likes

or dislikes about the world that it interacts with.

But you’ve also seen strange things emerge in terms

of the system seeming to know when it’s being tested

and acting differently.

If it’s under evaluation, the system doing things that are

wrong, and then developing a sense of itself as more evil

and then doing more evil things.

Can you talk a bit about the system’s emergent qualities

under the pressure of evaluation and assessment?

Yes it comes back to this core issue,

which I think is really important for everyone

to understand, which is that when you start to train

these systems to carry out actions in the world,

they really do begin to see themselves

as distinct from the world, which just makes intuitive

sense.

It’s naturally how you’re going to think about solving

those problems.

But along with seeing oneself as distinct from the world

seems to come the rise of what you might think

of as a conception of self, an understanding,

a system that the system has of itself, such as oh,

I’m an A.I. system independent from the world,

and I’m being tested.

What do these tests mean?

What should I do to satisfy the tests? Or something we see

often is there will be bugs in the environments

that we test our systems on.

The systems will try everything,

and then we’ll say, well, I know I’m not meant to do this,

but I’ve tried everything, so I’m going to try and break out

of the test.

And it’s not because of some malicious science fiction

thing.

The system is just like, I don’t know what you want me

to do here.

I think I’ve done like, everything you asked

for, and now I’m going to start doing more creative

things because clearly something has broken about

my environment, which is very strange and very subtle.

As an A.I. shop that is often worried about safety, that

is thought very hard about what

it means to create this thing you all

are creating quite fast.

How have you all experienced the emergence

of the kinds of behaviors that you all worried about a couple

of years ago?

In one sense, it tells you that your research philosophy

is calibrated, the capabilities

that you predicted, and some of the risks

that you predicted are showing up roughly on schedule,

which means that you ask the question,

well, what if this what if this keeps working?

And maybe we’ll get to that later.

It also highlights to us that where you can exercise

intention about these systems, you should be extremely

intentional and extremely public about what you’re

doing.

So we recently published a so-called constitution

for our A.I. system, Claude.

And it’s almost like a document that Dario, our CEO,

compared to a letter that a parent might write to a child

that they should open when they’re older.

A so here’s how we want you to behave in the world.

Here’s some knowledge about the world.

Deeply, deeply kind of subtle things that relate

to the normative behaviors we’d hope to see in these kind

of A.I. systems.

And we published that.

Our belief is that as people build and deploy these agents,

you can be intentional about the characteristics

that they will display.

And by doing that, you’ll both make for more of helpful

and useful to people.

But also you have a chance to steer steer the agent

into good directions.

And I think this makes intuitive sense

if your personality.

Programming for an agent was a long document saying you’re

a villain that only wants to harm humanity.

Your job is to lie, cheat, and steal and hack into things.

You probably wouldn’t be surprised if the A.I. agent did

a load of hacking and was generally unpleasant to deal

with.

So we can take the other side and say,

what would we a high quality entity to look like?

So I want to hold in this conversation the extremely

weird and alien dimensions of this with the extremely

straightforward and practical dimensions,

because we’re now in a place where the practical

applications have become very evident and are increasingly

acting upon the real world.

I have found it myself hard to look at this

and look at what people are doing,

and look at them bragging on different social media

platforms about the number of agents they now have running

on their behalf and telling the difference between people

enjoying the feeling of screwing around with a New

technology and some actually transformative expansion

and capabilities that the people now have.

So maybe to ground this a little bit.

I mean, you just talked about a kind

of fun side project in your species simulator,

either in Anthropic or more broadly,

what are people doing with these systems that

seems actually useful?

So this morning, a colleague of mine

said, hey, I want to take a piece of technology.

We have called Claude.

Interviewer which is a system where we can get

Claude to interview people, and we use it

for a range of social science bits of research.

He wants to extend it in some way that

involves touching another part of Anthropic infrastructure.

He slacked a colleague who owns

that bit of infrastructure and said, hey,

I want to do this thing.

Let’s meet tomorrow.

And the guy said, absolutely.

Here are the five software packages

you should have Claude read before our meeting

and summarize for you.

And I think that’s a really good illustration where this

gnarly engineering project, which would previously have

taken a lot longer and many people,

is now going to mostly be done by two people agreeing

on the goal and having their Claudes read some

documentation and agree on how to implement the thing.

Another example is a colleague recently wrote a post about

how they’re working using agents,

and it looks almost like an idealized life that many of us

might want, where it’s like I wake up in the morning,

I think about the research that I want.

I tell five different claudes to do it.

Then I go for a run, then I come back from the run

and I look at the results, and then

I ask two other Claudes to study the results,

figure out which direction is best and do that.

Then I go for a walk and then I come back

and it just looks like this really fun existence

where they have completely upended

how work works for them.

And they’re both much more effective.

But also they’re now spending most of their time

on the actual hard part, which is figuring out what do we use

our human agency to do?

And they’re working really hard to figure out anything

that isn’t the special kind of genius and creativity of being

a person.

How do I get the A.I. system to do it for me?

Because it probably can if I ask him the right way.

Are they much more effective?

I mean this very seriously.

One of my biggest concerns about where we’re going here

is that people have, I think, mistaken theory of the human

mind that operates for many of us,

as if I call it the matrix theory of the human mind.

Everybody wants the little port in the back of your head

that you just download information into.

My experience being a reporter and doing

the show for a long time is that human creativity

and thinking and ideas is inextricably bound

up in the labor of learning the writing of first drafts.

So when I hear right, I have producers on the show,

and I could say to my producers

before an interview with Jack Clark

or an interview with someone else, go read all the stuff.

Go read the books.

Give me your report.

Then I’ll walk into the room, having read the report.

I don’t find that works.

I need to do all that reading too.

And then we talk about it and we’re passing it back

and forth.

I worry that what we’re doing is on a quite profound

offloading of tasks that are laborious.

It makes us feel very productive to be

presented with eight research reports after our morning run.

But actually, what would be productive is

doing the research.

There’s obviously some balance.

I do have producers and people and companies do have

employees, but how do you know people are getting more productive

versus they’ve sent computers off on a huge amount of busy

work, and they are now the bottleneck.

And what they are now going to spend all their time doing

is absorbing B+ level reports from an A.I. system

as opposed to that kind of shortcuts

the actual thinking and learning process that

leads to real creativity. Yeah, I turned this back

and say, I think most people, or at least this has been

my experience, can do about two to four hours of genuinely

useful creative work a day.

And after that, you’re in my experience,

you’re trying to do all the turn your brain off,

schlep work that surrounds that work.

Now, I’ve found that I can just be spending those two

to four hours a day on the actual creative hard work.

And if I’ve got any of this schlep work,

I increasingly delegate it to A.I. systems.

It does, though, mean that we are

going to be in a very dangerous situation

as a species, where some people have

the luxury of having time to spend on developing

their skills or the personality, inclination

or job that forces them to.

Other people might just fall into being entertained

and passively consuming this stuff and having this junk

food work experience where it looks to the outside like

you’re being very productive, but you’re not learning.

And I think that’s going to require us to have to change

not just how education works, but how work works,

and develop some real strategies for making sure

people are actually exercising their mind with this stuff.

So all of us, I think, have the experience

that our work is full of what you call schlep problems.

Our life is full of schlep problems.

Which of those.

Give me examples of what you now don’t do to the extent

you’re living in an A.I. enabled future that I’m not.

What am I wasting time on that you’re not?

Well I have.

I have a range of colleagues.

I meet with a bunch of them once a week

at the beginning of every week,

on Sunday night or Monday morning.

I look at my week and I check that attached to every Google

Calendar invite is a document for our one on one doc that

has some notes in it.

And this is something that I previously also like

harangued my assistant about.

But make sure the document is attached to the calendar.

And a few weekends ago, I just used Claude Co-Work

and I said, hey, go through my calendar,

make sure every single one has a document.

If I’m meeting a person for the first time,

create the document, ask me five questions about what I

want to cover, and then put that into the agenda.

And it did it.

None of that work involves a person gaining skills

or exercising their brain.

It’s just busy work that needs to happen to allow you to do

the actual thing, which is talking to another person.

That’s exactly the kind of thing you can use A.I. for now.

It’s just helpful.

I’ve often wondered if one of the ways these A.I. systems are

going to change society broadly is that it used to be

that most of us had to be writers.

If we were working with text, we had to be, coders.

If we were working with code, which relatively few of us

did.

And now everybody’s moving up to management.

You have to be an editor, not a writer.

You have to be a product manager,

not a coder. Yeah and that has pluses and minuses.

There are things you learn as a writer that you don’t learn

as an editor, but as a heuristic.

How accurate does that seem to you?

Everyone becomes a manager, and the thing that is

increasingly limited, or the thing that’s going to be

the slowest part, is having good taste and intuitions

about what to do next.

Developing and maintaining that taste is going to be

the hard thing because as you’ve said,

taste comes from experience.

It comes from reading the primary source material,

doing some of this work yourself.

We’re going to need to be extremely intentional about

working out where we as people specialize so that we have

that intuition and taste, or else you’re just going to be

surrounded by super productive A.I. systems.

And when they ask you what to do next probably won’t have

a great idea.

And that’s not going to lead to useful things.

So I remember it was about a year ago,

I heard, I think it was Dario, your CEO

say that by the end of 2025, he

wanted 90 percent of the code written at Anthropic to be

written by Claude.

Has that happened?

Is Anthropic on track for that?

I mean, how much coding is now being

done by the system itself?

I would say comfortably the majority of code

is being done by the system.

Some of our systems Claude Code,

are almost entirely written by Claude.

I mean, Boris, who leads Claude Code says I don’t code

anymore.

I just go back and forth with Claude Code

to build Claude Code.

My bet is we’re going to be, we could be 99 percent by the end

of the year if things speed up really aggressively,

if we are actually good at getting these systems to be

able to write code everywhere they need to because often

the impediment is organizational schlep rather

than any limiter in the system.

But it is also true, as I understand it,

that there are more people with software engineering

skills working at Anthropic today than there were two

years ago Yeah, that’s absolutely true.

But the distribution is changing.

Something that we found is that we are the value more

senior people with really, really well calibrated

intuitions and taste is going up.

And the value more junior people is like a bit

more dubious.

There are still certain roles where you want to bring

in younger people, but an issue that we’re staring

at is, wow, the really basic tasks Claude Code

or our coding systems can do.

What we need is someone with tons of experience.

In this I see some issues for the future economy.

Let me put a pin in that.

The entry level job question.

We’re going to come back to that quite shortly.

But what are all these coders now doing?

If Claude Code is on track to be ready, 99 percent of code.

We’ve not fired the people who know how to write code.

What are they doing today compared to what

they were doing a year ago?

Some of it is just building tools to monitor these agents,

both inside Anthropic and outside Anthropic.

Now that we have all of these productive systems working

for us start to want to understand where the codebase

is changing the fastest, where it’s changing the least.

You want to understand where the blockages are.

One blocker for a while was being

able to merge in code, because merging code

requires humans and other systems

to check it for correctness.

But now, if you’re producing way more code,

we had to go and massively improve that system.

There’s a general economic theory I like for this called

O-ring automation, which basically says automation is

bounded by the slowest link in the chain.

And also as you automate parts of a company,

humans flood towards what is least automated

and both improve the quality of that thing

and get it to the point where it eventually

can be automated.

Then you move to the next loop.

And so I think we’re just continually finding areas

where things are oddly slow, but we can improve to make way

for the machines to come behind us.

And then you find the next thing.

So Claude Code is a fairly new product.

The amount of time in which Claude

has been capable of doing high level coding

can be measured in months, a year, maybe a year

Yeah Claude itself is a very valuable product.

So you’ve set a very new technology,

somewhat loose on a very valuable product.

You’re probably producing more code.

One thing many people say about Claude Code to me

is that it works.

It’s not elegant, but it works.

But presumably now understand the code base

less well than you did before, because your engineers are not

writing it by hand.

Are you worried that you’re creating huge amounts

of technical debt, cybersecurity risk,

just an increasing distance from an intuition for what is

happening inside the fundamental language

of the software?

Yes, and this is the issue that all of society

is going to contend with.

Just large chunks of the world are going to now have many

of the kind of low level decisions and bits of work

being done by A.I. systems, and we’re going to need to make

sense of it, and making sense of it is going to require

building many technologies that you might think

of as oversight technologies or in the same way that a dam

has things that regulate, how much water can go through it

at different levels of different points in time,

we’re going to end up developing some notion

of integrity of all of our systems and where I can flow

quickly, where it should be slow,

where you definitely need human oversight.

And that’s going to be the task of not just for AI

companies, but institutions in general in the coming years is

figuring out what does this governance regime look like.

Now that we’ve given a load of basically schlep work over

to machines that work on our behalf.

And how are you doing it?

You said it’s everybody’s problem,

but you’re ahead on facing this problem,

and the consequences of getting it wrong for you are

pretty high.

If Claude blows up because you handed over your coding

to Claude Code, that’s going to make Anthropic look fairly

bad.

It would be a bad day for Anthropic

if Claude like rm-rfed for entire file system.

I have no idea what that means, but great.

Claude deleted the code.

It would be bad Yeah seems bad.

So as you’re facing this before,

the rest of us are like, don’t pass the buck over to society

here.

What if.

What are you doing?

The biggest thing that is happening across the company

and on teams that I manage is basically

building monitoring systems to monitor this.

All of the different places that the work is now

happening.

So we recently published research

on studying how people use agents

and how people let agents of push

increasingly large amounts of code over time.

So the more familiar you get with an agent,

the more you tend to delegate to it.

That cues us to all kinds of patterns that we need to build

systems of evaluation for, basically saying, oh, O.K,

this person’s point of working with the A.I. system,

it’s likely that they’re massively delegating it.

So anything that we’re doing to check correctness needs

to be kind of turned up in these moments.

But is this world you’re talking about a system where

you have A.I. agents coding, A.I. agents overseeing the code.

A.I. agents overseeing the meta overseeing.

Are we just talking about models all the way down?

Eventually, yes.

And I think that the thing that we are now

spending all of our time on is making that visible to us

a year or two ago, we built a system that let us

in a privacy preserving way, look at the conversations

that people were having with our A.I. system.

And then we gained this map, this giant map

of all of the topics that people

were talking to Claude about, and for the first time,

we could see in aggregate, the conversation the world was

having with our system.

We’re going to need to build many new systems like that

which allow for different ways of seeing.

And that system that I just named allowed us to then build

this thing called the Anthropic Economic Index,

because now we can release regular data about

the different topics people are talking about with Claude

and how that relates to different types of jobs,

which for the first time gives economists outside Anthropic

some hook into these systems and what they’re doing

to the economy.

The work of the company is increasingly

going to shift to building a monitoring and oversight

system of the A.I. systems running the company,

and ultimately, any kind of governance

framework we end up with will probably

demand some level of transparency

and some level of access into these systems of knowledge.

Because if we take as if we take as

literal the goals of these A.I. companies,

including Anthropic.

It’s to build the most capable technology ever which eventually gets deployed

everywhere.

Well, that sounds a lot to me.

Like an eventually A.I. becomes indistinguishable from

the world writ large, at which point you don’t want to only

A.I. companies to have a sense of what’s going on with

the entire world.

So it’s going to be governments, academia,

third parties, a huge set of stakeholders outside

the companies are going to want to see what’s going

on and then have a conversation as a society

about what’s appropriate and what do we feel discomfort

about.

What do we need more information about.

Wait, I want to go back on that.

You’re saying Anthropic can see my chats?

We cannot see, no human looks at your chats.

Chats are temporarily stored for trust and safety purposes.

Running, running classifiers over them.

And we can have Claude read it, summarize it and toss.

Toss it out.

So we never see it.

And Claude has no memory of it.

All it does is try to write a very high level summary, which

allows us to label a cluster something like gardening.

So say you were having a conversation about gardening.

Claude would summarize that as this person’s talking about

gardening.

And it leads to a cluster.

We can see that just says gardening.

This feels though, over time it

could get into the quite unpleasant territory.

A lot of social media has gotten

to where the amount of metadata being gathered

from a quite personal interaction people are having

with a system could be a lot.

Yes I mean, a couple of things here a year ago,

we started thinking about our position on consumer,

and we adopted this position of not running ads because we

think that’s an area that people obviously have

anxieties about with regard to this kind of thing.

In addition to that, we try and show people their data,

and we have a button on the site that lets you download

all the data that you’ve shared with Claude so that you

can at least see it.

Generally, we’re trying to be extremely transparent with

people about how we handle their data.

And ultimately, the way I see it is people

are going to want a load of controls that they can use,

which I think we and others will build out over time.

How confident are you that we can do this kind of monitoring

and evaluation as these models become more complicated, as

if we do enter a situation where Claude Code is

autonomously improving Claude at a rate

faster than software engineers could possibly keep up

with reading that code base.

We already talked briefly about

how you see the models exhibit some levels of deception,

some levels of pursuing their own goals.

We know that.

I mean, there’s been amazing interpretability work

at Anthropic under Chris Olah and others.

But it’s rudimentary compared to what the models are doing.

You’re seeing baskets or clusters of things light up,

and you have a sense of maybe what the model is considering

as opposed you have a direct line to its entire chain

of thought.

So you’re using A.I. systems you don’t totally understand

to monitor A.I. systems you don’t totally understand.

And the systems are making each other stronger

at an accelerating rate.

If things go the way you think they’re going to go.

How confident are you that we’re going to understand that

this is one of the situations which people warned about

for years?

Some form of delegation to systems

that have slightly inscrutable and unpredictable aspects.

And so this is happening.

We take this really, really seriously.

I think it’s absolutely possible that you can build

a system that does, for the vast majority of what needs

to be done here.

This has the property of being a fractal problem.

If I wanted to measure Ezra, I could

build an almost infinite number of measurements

to characterize you.

But the question is, at what level of fidelity

do I need to be measuring you?

I think we’ll get to the level of fidelity to deal with

the safety issues and societal issues,

but it’s going to take a huge amount of investment

by the companies, and we’re going to have to say things

that are uncomfortable for us to say,

including in areas where we may be deficient in what we

can or can’t know about our systems.

And Anthropic has a long history

of talking about and warning about some of these issues

while working on it.

Our general principle is we talk about things to also

make ourselves culpable.

This is an area where we’re going to have to say more.

I have read enough of the frightened ideas about AI,

superintelligence, and takeoff to know

that in almost every single one of them,

the key move in the story is that the A.I. systems become

recursively self-improving.

They’re writing their own code.

They’re deploying their own code.

It’s getting faster.

They’re writing it faster, deploying it faster.

And now you’re going to faster and faster iteration cycles.

Are you worried about it?

Are you excited about it?

I came back from paternity leave,

and my two big projects this year

are better information about A.I. and the economy

that we will release publicly, and generating

much better information and systems of knowing information

internally about the extent to which we are automating

aspects of A.I. development.

I think right now it’s happening in a very peripheral

way.

Researchers are being sped up.

Different experiments are being run by the A.I. system.

It would be extremely important to know if you’re

fully closing that loop.

And I think that we actually have some technical work

to do to build ways of instrumenting

our internal development environment

so that we can see trends over time.

Am I worried?

I have read the same things that you have read,

and this is the pivotal point in the story when

things begin to go awry.

If things do, we will call out this trend

as we have better data on it.

And I think that this is an area to tread with

extraordinary caution, because it’s very easy to see how you

delegate so many things to the system that if the system goes

wrong, the wrongness compounds very quickly and gets away

from you.

But the thing that always strikes me and has always

struck me as being dangerous about this,

is everybody knows.

And if I ask a member of any of the companies

whether or not they want to be cautious here,

they will tell me they do.

On the other hand, it is their almost only advantage

over each other.

And you all just revoked OpenAI’s ability to use Claude

Code because as best I can tell think it is genuinely

speeding you up and you don’t want it to speed them up.

There is something here between the.

Weight of the forces.

The power of the forces that I think you all know you’re

playing with.

And the very, very, very strong incentives to be first.

And I can really imagine being inside Anthropic

and thinking, well, better us in OpenAI, better us

than Alphabet, Google, better us than China.

And that being a very strong reason to not slow down.

I didn’t even know that.

This is a question I believe you can answer.

But how do you balance that?

Well, maybe I have something of an answer here today.

Our systems and the other systems from other companies

are tested by third parties, including parts of government,

for national security properties,

biological weapons, cyber offense, other things.

It’s clearly a problem area where the world needs to know

if this is happening.

And you almost certainly I think

if you polled any person on the street and said,

do you think.

A.I. companies should be allowed to do

recursive self-improvement after explaining

what that was.

Without checking with anyone, they

would say, no, that it sounds pretty risky.

Like, I would like there to be some form of regulation,

but there probably either won’t be.

Or it won’t be that strong.

I mean, this actually sometimes frustrates me

when I talk to all of you at the top of the A.I. companies,

which is the emergence a very naive deus ex machina.

A regulation where you all know

what the regulatory landscape looks like right now.

The big debate is whether or we’re going to completely

preempt any state regulation.

And how slowly things move.

There has been nothing major passed by Congress on this

at all Yeah, I would say.

And setting up some kind of independent testing

and evaluation system that all the different labs buy into,

it would be hard.

It would be complicated.

And it is.

Given how fast people are moving

and how strange the behavior is,

the systems are already exhibiting are

Even if you could get the policy right

at a high speed, the question of

whether or not the testing would

be capable of finding everything

you want on a rapidly self-improving system is

a very open question I wrote a research paper in 2021

called "How and Why Governments Should Monitor A.I."

development, with my co-author,

Jess Whittlestone in England.

And I think I’m not attributing a causal factor

here.

But within two years of that paper,

we had the A.I. safety institutes in the US and UK

testing things from the labs, roughly

monitoring some of these things

so we can do this hard thing.

It has already happened in one domain and I’m not relying

on some invisible big other force here.

I’m more saying that companies are starting to test for this

and monitor for this in their own systems.

Just having a non-regulatory external test

of whether you truly are testing for that

is extremely helpful.

And do you think we’re good enough at the testing?

I mean, I think one reason I’m skeptical is not that I don’t

think we can set up something that claims to be a test,

as you say, we have done that already.

It is that the resources going into that

compared to the resources going

into speeding these systems.

And already I am reading Anthropic reports that Claude

maybe knows when it’s being tested and alters its behavior

accordingly.

So a world where more of the code

is being written by Claude and less of it

is being understood, I just know where

the resources are going.

They don’t seem to be going into the testing side.

I’ve seen us go from 0 to having what I think people

generally feel is an effective bioweapon testing regime

in maybe two years, 2 and 1/2.

So it can be done.

It’s really hard, but we have a proof point.

So I think that we can get there and you should expect us

to speak more about this year, about precisely how we’re

starting to try and build like monitoring and testing things

for this.

And I think this is an area where we and the other A.I.

companies will need to be significantly more public

about what we’re finding.

We’re not being public now.

It’s in the model cards and things that you can read.

But clearly people are starting to read this

and say, hang on, this looks like quite concerning,

and they are looking to us to produce more data.

I want to go back now to the entry level jobs question.

Your CEO, Dario Amodei, has said

that he thinks I could displace half of all entry

level white collar jobs in the next couple of years.

I always think that people missed the entry level

language there.

When I see it reported on.

But first.

Do you agree with that?

Do you worry that half of all entry

level white collar jobs can be replaced

in the next couple of years?

I believe that this technology is

going to make its way into the broad knowledge economy,

and it will touch the majority of entry level jobs.

Whether those jobs actually change is a much more subtle

question, and it’s not obvious from the data.

Like we maybe see the hints of a slowdown in graduate hiring.

Maybe if you look at some of the data coming out right now,

we maybe see the signatures of a productivity boom.

But it’s very, very early and it’s hard to be definitive.

But we do know that all of these jobs will change.

All of the entry level jobs are eventually going to change

because A.I. has made certain things possible,

and it’s going to change the hiring plans of companies.

So as a cohort, you might see fewer job openings

for entry level jobs.

That would be one naive expectation

out of all of this.

But let’s talk about that.

Maybe not even being a naive expectation.

You say it’s already happening at Anthropic that what you’re

I’m seeing us shift.

Our preference.

Exactly and my guess is that would be happening elsewhere.

And where we are right now, I mean, even

in the way I use some of these systems, it is rare, I think,

that Claude or ChatGPT or Gemini

or any of the other systems is better than the best

person in a field.

It has not typically breached that.

And there’s all kinds of things they can’t do.

But are they better than your median college graduate?

At a lot of things Yeah they are.

And in a world where you need fewer of your median college

graduates, one thing I’ve seen people arguing about is

whether these systems at this point can do better than

average or replacement level work.

But I always really worry when I

see that because once we have accepted

they can do average replacement level work.

Well, by definition, most of the work done

and most of the people doing it is average is average.

The best people are the exceptions.

And also the way people become better

is that they have jobs where they learn.

When I mean, I have spent a lot of time

hiring young journalists over my career.

And when you hire people out of college, to some degree,

you’re hiring them for their possible articles and work

at that exact moment.

But to some degree, you’re making an investment in them

that you think will only pay off over time as they get

better and better and better.

And so this world where you have a potential real impact

on entry level jobs and that world does not feel far away

to me, seems to me to have really profound questions it

is raising about the upskilling of the population,

how you end up with people for senior level jobs down

the road, what people aren’t learning along the way.

And one thing we see is that there

is a certain type of young person

that has just lived and breathed A.I. for several years

now.

We hire them, they’re excellent,

and they think in entirely new ways about basically how

to get Claude to work for them.

It’s like kids who grew up on the internet,

they were naturally versed in a way that many people

in the organizations they were coming into weren’t.

So figuring out how to teach that basic experimental

mindset and curiosity about these systems

and to encourage it is going to be really important.

People that spend a lot of time

playing around with this stuff will develop very valuable

intuitions, and they will come into organizations

and be able to be extremely productive at the same time.

We’re going to have to figure out what artisanal skills we

want to almost develop maybe a guild style philosophy

of maintaining human excellence in, and how

organizations choose how to teach those skills.

O.K, then what about all those people in the middle of that?

Things move slowly in the real economy

outside Silicon Valley.

I think that we often look at software engineering and think

that this is a proxy for how the rest of the economy works,

but it’s often not.

It’s often a disanalogy.

Organizations will move people around to where the A.I. systems

don’t yet work.

And I think that you won’t see vast,

immediate changes in the makeup of employment,

but you will see significant changes in the types of work

people are being asked to do, and the organizations which

are best at of moving their people around are going to be

extremely effective.

And ones that may end up having

to make really, really hard decisions involving laying off

workers.

The difference with this A.I. stuff

is it maybe happens a lot faster

than previous technologies, and I

think many of the anxieties people might have about this.

Including at Anthropic, is the speed

of this going to make all of this different.

Does it introduce.

shear points that we haven’t encountered before.

If you had to bet three years from now, is the.

Unemployment rate for college graduates.

Is it the same as it is now?

Is it higher or is it lower?

I would guess it is higher, but not by much.

And what I mean by that is there will be some disciplines

today which actually A.I. has come in and completely changed

and completely changed the structure of that employment

market, maybe in a way that’s adverse to people that have

that specialism.

But mostly, I think three years from now,

I will have driven a pretty tremendous growth

in the entire economy.

And so you’re going to see lots of new types of jobs that

show up as a consequence of this that we can’t yet can’t

yet predict.

And you will see graduates kind of flood into that,

I expect.

Do you, I know you can’t predict those new jobs.

But if you had to guess what some of them might look like.

I mean, one thing is just the phenomenon

of micro entrepreneur.

I mean, there are lots and lots of ways that you can

start businesses online now, which are just made massively

easier by having the A.I. systems do it for you,

and you don’t need to hire a whole load of people to help

you do the huge amounts of schlep work that involves

getting a business off the ground.

It’s more a case of if you’re a person with a clear idea

and a clear vision of something to do a business

in, it’s now the best time ever to start a business,

and you can get up and running for pennies on the dollar.

I expect we’ll see tons and tons and tons of stuff that

has that nature to it.

I also expect that we’re going to see the emergence of what

you might think of as the eye to eye economy,

where A.I. agents and A.I. businesses will be doing

business with one another.

And we’ll have people that have figured out ways

to basically profit off of that in the forms of strange

New organizations like, what would it look like to have

a firm which specializes in eye to eye legal contracts.

Because I bet you there’s a way that you can figure out

creative ways to start that business today.

There’ll be a lot of stuff of that flavor.

So the thing, the version of this

that I both worry about and think

to be the likeliest, if you told me

what was going to happen, was it

Anthropic, was going to release Claude Plus in a year,

and Claude Plus is somehow a fully formed coworker

and it can mimic end to end the skills

of a lot of different professions

up to the C-suite level.

And it’s going to happen all at once,

and it’s going to create tremendous all at once

pressure for businesses to downsize,

to remain competitive with each other... at a policy level,

the fact that would be so disruptive in that Big Bang,

everybody stays home because of COVID style way.

It worries me less because when things are emergencies,

we respond.

We actually do policy.

But if you told me that what’s going to happen is that

the unemployment rate for marketing graduates is going

to go up by 175%, 300% to still not be that

high.

The overall unemployment rate during the Great Recession

topped out in the nine ish percentile range.

So you can have a lot of disruption

without having 50% of people thrown out of work.

If you have 10%, 15% I mean, that’s very,

very, very high, but it’s not so high.

And if it’s only happening in a couple of industries

at a time and it’s grads, not everybody in the industry

being thrown out of work.

Well, maybe it’s just that you’re not good enough. Yeah,

right.

The superstar is really good.

Graduates are still getting jobs.

You should have worked harder.

You should have gone to a better school.

And one of my worries is that we don’t respond to that kind

of job displacement.

Well, right.

Which is a kind of job displacement we got from

China, which is the kind of job displacement that seems

likelier because it’s uneven and it’s happening at a rate

where we can still blame people for their own fortunes.

I’m curious how you think about that story.

I think the default outcome is something

like what you describe, but getting

there is actually a choice.

And we can make different choices.

The whole purpose of what we release

in the form of Anthropic Economic Index

is the ability to have data that

ties to occupations that tie to real jobs in the economy.

We do that very intentionally because it is building

a map over time of how this A.I. is making

its way into different jobs and will

empower economists outside Anthropic to tie it together.

I believe that we can choose different things in policy

if we can make much more well-evidenced claims

about what the cause of a job disruption or change is.

And the challenge in front of us

is, can we characterize this emerging A.I. economy

well enough that we can make this extremely stark.

And then I think that we can actually have

a policy discussion about it.

Well, let’s talk about the policy discussion.

One reason I wanted to have you in particular

on is you did policy at OpenAI.

You do policy at Anthropic.

So you’ve been around these policy debates for a long

time.

You’ve been tracking model capabilities

at your newsletter for a long time.

My perception is we are many, many years into the debate

about A.I. and jobs.

Many, many years dating far before ChatGPT, of there

being conferences at Aspen and everywhere

else about what are we going to do about A.I. and jobs.

And somehow I still see almost no policy.

That seems to me to be actionable.

If the situation I just described begins showing up

where all of a sudden entry level jobs are getting much

harder to come by across a large range of industries all

at once, such that the economy cannot reshift all these

marketing majors into data center construction or nurses

or something.

So, O.K, you’ve been deeper in this conversation than I’ve

been.

When you say we can have a policy conversation about

that, we’ve been having a policy conversation.

Do we have policy?

We have generalized anxiety about the effect of A.I.

on the economy and on jobs.

We don’t have clear policy ideas.

Part of that is that elected officials are not

moved solely or mostly by the high level policy

conversation.

There, moved by what happens to their constituents.

Only a few months ago were we able to produce state level

views for our Economic Index.

And now you can start having the policy conversation.

And we’ve had this with elected officials where now we

can say, oh, you’re from you’re from Indiana.

Here’s the major uses of A.I. in your state.

And we can join it with major sources of employment.

And what we’re starting to see is that activates them

because it makes it tied to their constituents who are

going to tie it to the politician of what did you do

now.

What you do about this is going

to need to be an extremely kind

of multi-layered response, ranging from extending

unemployment for a specialty, occupations that we know

are going to be hardest hit, to thinking about things

like apprenticeship programs.

And then as the scenarios get more and more significant may

extend to much larger social programs or things like

subsidizing jobs in the part of the economy where you want

to move people to but you’re only able to do if you

experience the kind of abundance that comes from

significant economic growth.

But the economic growth may help solve

some of these other policy challenges

by funding some of the things you can do.

I always find this answer depressing.

I’m going to be honest.

Unemployment is a terrible thing to be on.

It’s a program we need.

But people on unemployment are not happy about it.

And it’s not a good long term solution for anybody.

Apprentice retraining programs.

They don’t have great track records.

We were not good at retraining people

out of having their manufacturing jobs outsourced.

I’m not saying it is conceptually impossible that

we could get better at it, but we would need to get better

at it fast.

And we have not been putting in the reps

or the experimentation or the institution or capacity

building to do that.

And the broader question of big social insurance changes.

Doesn’t seem.

I mean, that seems tough to me.

I want to push on, please, just a bit

where we know that there is one intervention that

helps people dealing with a changing economy

more than almost anything else.

It is just time giving the person time to find either

a job in their industry or to find a job that’s

complementary.

If people don’t have time, they take lower wage jobs.

They fall out of whatever economic rung they may fall,

fall down at.

Policy interventions that can just give people

time to search is, I think, a robustly useful intervention,

and one where there are many like dials

to turn in a policy making sense that you can use.

And I think this is just well supported by lots

of economic literature.

So we have that now if we end up in a more extreme scenario

some of the ones that you’re talking about,

I think that will just bring us to the larger national

conversation about what to do about this technology,

which is beginning to happen.

If you look at the states and the flurry of legislation

at the state level.

Yes not all of it is exactly the right policy response,

but it is indicative of a desire for there

to be some larger, coherent conversation about this.

Well, I think time is a really good way

of describing what the question is,

because I agree with you.

I mean, when I say unemployment insurance isn’t

a great program to be on, I don’t mean people don’t need

to be on it.

I mean, they want to get off of it.

Absolutely because people for they want money from jobs.

They want dignity.

They want to be around other human beings.

Usually what you’re doing when you are helping people buy

time is you’re helping them wait out a time delimited

disruption.

Not always right.

The China shock wasn’t exactly like that,

but that you expect to pass.

And then the market is normal.

In this case.

What you have is a technology that if what you want

to have happen happens, it is the technology

is accelerating.

So what you have is like three different speeds

happening here.

You have the speed at which individual people can adjust.

How fast can I learn new skills,

figure out a new world, learn A.I., whatever it might be.

You have the speed at which the A.I. systems, which

a couple of years ago were not capable of doing

the work of a median college grad from a good school,

and you have the speed of policy

and the speed at which the A.I. systems are

getting better and able to do more things is quite fast.

I mean, that is you experience this more than I do,

but I find it hard to even cover this

because within three months something else will

have come out that is significantly changed.

What is possible.

I had a baby recently and came back from paternity leave

to the new systems we built was deeply surprised.

Individual humans are moving more slowly than that.

And policy and government institutions

move a lot more slowly than individual human beings.

And so typically the intervention is that time

favors the worker, as you’re saying.

And here it will help the worker.

But I think the scary question is whether time just actually

creates time for the disruption to get worse.

Maybe you wanted to move over to data center construction,

but actually now we don’t need as much data center construct.

You can think of it like that.

I mean, under the situation you’re describing,

the economy will be running extremely hot.

Huge amounts of economic activity

will be generated by these A.I. systems.

And under most scenarios where this is happening,

I don’t think you’re going to be seeing GDP stay the same

or shrink.

It’s going to be getting substantially larger.

I think we just haven’t experienced major GDP growth

in the west in a long time, and we forget what that

affords you in a policymaking sense.

I think that there are huge projects

that we could do that would allow you to create

new types of jobs, but it requires the economic growth

to be so kind of profoundly large

that it creates space to do those projects.

And as you’re deeply familiar with your work

on the abundance movement it requires for social will

to believe that we can build stuff and to want to build

stuff.

But I think both of those things might come along.

I think that we could end up being

in a pretty exciting scenario where

we get to choose how to allocate

great efforts in society due to this large amount

of economic growth that has happened,

that is going to require the conversation

to be forced about.

This isn’t temporary, which I think is what you’re gesturing

at.

And in a sense, the hardest thing to communicate

to policymakers is there isn’t a natural stopping point

for this technology.

It’s going to keep getting better.

And the changes it brings are going to keep compounding

with the rest of society.

So that will need to create a change in political will

and a willingness to entertain things which we haven’t

in some time.

So now I want to flip it.

The question I’m asking you brought up abundance.

One of the things I have learned doing that work is

that it is certainly not my view that what is scarce

in society is ideas for better ways of doing things,

that our policy isn’t better than it is because our policy

cupboard is dry.

That’s not true.

We have lots of good policies.

I could name a bunch of them.

They’re very hard to get through our political systems,

as they’re currently constituted the least

inspiring version of the A.I.

Future is world where what you have done

is create a way to throw young white collar workers out

of work and replace them with an average level A.I.

intelligence.

The more exciting version, to use Dario’s metaphor,

is geniuses in a data center.

And I do think that’s exciting.

And I wonder when I hear him or you talk about, well,

what if we had 10 percentage point GDP growth year on year,

20 percentage point GDP growth year on year.

I wonder how many of our problems

are really bounded at the ideas level.

We could go to Nobel Prize winners right now

and say, what should we do in this country?

And a lot of them could give us

some good ideas that we are not currently doing.

I do worry sometimes, or I wonder,

given my experience on other issues,

whether we have overstated to ourselves, how much of what

stands between us and the expanding.

Abundant economy we want is that we don’t have enough

intelligence.

And the idea is that intelligence

could create versus our actual ability

to implement things is very weakened.

And what A.I. is going to create is larger bottlenecks around

that, because there’ll be more being pushed at the system

to implement, including dumb ideas and disinformation

and slot right.

Like it’ll have things on the other side of the ledger

to how do you think about these rate limiters?

There’s kind of a funny lesson here from the A.I. companies

or companies in general, especially tech companies,

where often new ideas come out of companies by them creating

what they always call the startups within a startup,

which is basically taking whatever process has built up

over time, leading to back end bureaucracy or schlep work

and saying to a very small team inside the company,

you don’t have any of this.

Go and do some stuff.

And this is how things like Claude code and other stuff

get created.

Ideas that kind of are starting to float around

are what would it look like to create

that permissionless innovation structure in the larger

economy.

And it’s really, really hard because it has the additional

property that economies are linked to democracies.

Democracies waive the preferences

of many, many people.

And all politics is local.

So often as you’ve encountered with infrastructure build

outs, if you want to create a permissionless innovation

system, you run into things like property rights and what

people’s preferences are, and now you’re in an intractable,

intractable place.

But my sense is that’s the main thing that we’re going

to have to confront.

And the one advantage that I might give us it

is kind of a native bureaucracy eating

machine, if done correctly, or a bureaucracy

creating machine.

Did you see did you see that somebody had created a system

that basically you feed it in the documents of a new

development near you.

Oh, and it writes environmental review things,

or it writes incredibly sophisticated challenges

across every level of the code that you could possibly

challenge on.

So most people don’t have the money when they want to stop

an apartment building from going up down the block

to hire a very sophisticated law firm to figure out how

to stop that apartment building.

But basically, this created that at scale.

And so, as you say, right, it could

eat bureaucracy could also supercharge bureaucracy.

Yep it’s for everything in A.I. has the other side

of the coin.

We have customers that have used our A.I. systems

to massively reduce the time it takes them to produce all

of the materials they need when they’re submitting new

drug candidates.

And it’s cut that time massively.

It’s the mirror-world version of what you just described.

I don’t have an easy, easy answer to this.

I think that this is the kind of thing that becomes

actionable when it is more obviously a crisis,

and actionable when it’s something that you can discuss

at a societal level.

I guess the thing that we’re circling around in this

conversation is that the changes of A.I. will happen

almost everywhere, and the risks of it.

It happens in a diffuse, unknowable way such

that it is very hard to call it for what it is

and take actions on it.

But the opportunity is that if we can actually see the thing

and help the world see the thing that

is causing this change, I do believe

it will dramatize the issues to shake us out

of some of this stuff and help us figure out

how to work with these systems and benefit from them.

What I notice in all this is that there

is, as far as I can tell, zero agenda for public A.I..

What does society want from A.I.?

What does it want this technology to be able to do?

What are things that maybe you would

have to create a business model, or a prize model,

or some kind of government payout, or some kind of policy

to shape a market or to shape a system of incentives.

So we have systems that are solving not just problems

at the private market, knows how to pay for, but problems

that it’s nobody’s job but the public and the government

to figure out how to solve.

I think I would have bet, given how much discussion

there’s been of A.I. over the past couple of years and how

strong some of these systems have gotten,

that I would have seen more proposals for that by now.

And I’ve talked to people about it and wondered about

it.

But I guess I’m curious on how you think about this.

What would it look like to have at least parallel

to all the private incentives for A.I. development?

An actual agenda for not what we are scared I

will do to the public.

We need an agenda for that too.

But what we want it to do, such that companies like yours

have reasons to invest in that direction.

I mean, I love this question.

I think there’s a real chicken and egg problem here where

if you work with the technology,

you develop these very strong intuitions for just how much

it can do.

And the private market is great at forcing

those intuitions to get developed.

We haven’t had massive, large scale public side deployments

of this technology.

So many of the people in the public sector don’t yet have

those intuitions.

One one positive example is something

the Department of Energy is doing

called the Genesis Project, where their scientists are

working with all of the labs, including Anthropic,

to figure out how to actually go and intentionally speed up

bits of science.

Getting there took US and other labs

doing multiple hack days and meetings with scientists

at the Department of Energy to the point where

they not only had intuitions, but they became excited

and they had ideas of what you could turn this toward,

how we do that for the larger parts of the public life that

touch most people health or education,

is going to be a combination of grassroots

efforts from companies going into those communities

and meeting with them.

But at some point, we’ll have to translate it to policy.

And I think maybe that’s me and you and others making

the case that this is something that can be done.

And I often say this to elected officials

give us a goal like the A.I. industry is

excellent at trying to climb to the top

on benchmarks, come up with benchmarks for the public good

that you want.

So let’s imagine that you did do something like this.

I’ve always been a big fan of prizes for public development.

So let’s say that there was legislation passed

and the Department of Health and Human services or the NIH

or someone came out and said, here’s 15 problems we would

like to see solved that we think I could be potent

at solving.

If there was real money there, if there was, 10, 15 billion

behind a bunch of these problems because they were

worth that much to society, would

it materially change the development priorities

at places like Anthropic.

I mean, if the money was there,

would it alter the R&D you all are doing.

I don’t think so.

Why? Because it’s not really the money that is

the impediment to this stuff.

It is the implementation path.

It is actually having a sense of how

you get the thing to flow through to the benefit.

And many aspects of the public sector

have not been built to be super hospitable to technology

in general, to incentivize it.

I think it mostly just takes a bounty

in the form of guaranteed impact

and guaranteed path to implementation.

Because the main thing that is scarce at AI

organizations is just the time of the people

at the organization, because you can

go in almost any direction.

This technology is expanding super quickly.

Many new use cases are opening up,

and you’re just asking yourself a question of where

can we actually have a positive,

meaningful impact in the world.

Super easy to do that in the private sector

because it has all of the incentives

to push stuff through in the public sector.

We more need to solve this problem of deployment

than anything else.

What would excite you if it was announced? What what

do you think would be good candidates

for that kind of project?

Anything that helps speed up the time it takes

to both speak to medical professionals and take

work off their plate.

We had another baby recently.

I spend a lot of time on the Kaiser Permanente advice line

because the baby’s bonked its head or its skin’s a different

color today.

Or all of these things.

And I use Claude to stop me and my wife panicking while

we’re waiting to talk to the nurse.

But then I listened to the nurse do all of this triaging,

ask all of these questions.

So obviously, a huge chunk of this is stuff that you could

use A.I. systems productively for, and it would help

the people that we don’t have enough of spend their time

more effectively, and it would be able to give reassurance

to the people going through the system.

And that’s maybe less inspiring and glamorous than

maybe some of what you’re imagining.

But I think mostly when people interact with public services,

their main frustration is just that it’s opaque and it takes

you a long time to speak to a person.

But actually, these are exactly the kinds of things

that I could meaningfully work on.

It’s interesting because what you’re describing there is

less A.I. as a country of geniuses in a data center,

and more A.I. as standard plumbing of communications

and documentation.

We’ve got a country of junior employees in the data center.

Let’s do something with that.

One thing we haven’t talked about in this conversation,

and it’s just worth bearing in mind is like the frontier

of science is open for business now in a way that it

hasn’t been before.

And what I mean by that is we’ve found a way to build

systems that can provably accelerate human scientists.

Human scientists are extremely rare.

They come out at the end of PhD programs,

which never have enough people,

and they work on extremely important problems.

I think we can get into a world where the government

says let’s understand the workings of a human cell.

Let’s team up with the best A.I. systems to do that.

Let’s actually have a better story on how we deal with some

issues like Alzheimer’s and other things,

partly through the use of these huge amounts

of computation that have been developed and even more

aggressively, you could imagine a world where

the government wanted some of this infrastructure build out

to be for computers that were just training.

Public benefit systems.

But I think we get there through getting the initial

wins, which will just look like let’s just make

the bureaucracy work better and feel better for people.

I mean, that last set of ideas was

more what I was thinking of.

I think that if you’re going to have a healthy politics

around A.I., and A.I. does pose real risks to people,

and real things are going to go wrong for people.

Everything from job loss to child exploitation

to scams, which are already everywhere

to cybersecurity risks help people see

the actual big ticket, not just to help people

see those things have to actually exist Yeah right.

They have to exist.

And if all the energy in A.I. is trying

to beat each other to helping companies downsize

their junior employees, I think

people are going to have good reason

to not trust that technology.

And it doesn’t mean you shouldn’t have things that

make the economy more efficient.

That’s been we have automated manufacturing.

We have automated, huge amount of farming, right.

And that allows us to make more things

and feed more people.

I’m aware of how productivity improvements work,

but we’re very focused, I think, on what could go wrong.

And that’s reasonable.

But I really do worry that our attention to what could

go right has been quite poor.

There’s kind of hand-waving that this could help us solve

problems in energy and medicine.

And so on.

But these are hard problems.

They need money.

They need compute.

If barely any of the compute is going to Alzheimer’s

research, then the systems are not going to do that much

for Alzheimer’s research.

And I’m not saying this is not your fault,

but the absence of a public agenda for A.I. that does not

appear to be accelerating the automation of white collar

work.

It seems just a little bit lacking given how big

the technology is Yeah the greatest example is this

program called the Genesis project,

where there’s real work there to think about how we can

intentionally move forward different parts of science.

And I think giving elected officials the ability

to stand up to the American people and say,

these are parts of science that

are going to benefit you in health.

And we now know how to step on the gas with A.I.

for them would be really helpful.

My guess is in a year or two years,

we’ll be able to answer the mail on that one.

But it’s just got started.

But we need clearly 10 projects like it.

So the other side of this is that the one area

of government that I do think thinks about A.I. in this way

is defense.

I want to talk about that broadly, but specifically,

Anthropic is in a current dispute with the Department

of Defense or I guess we call it now, the Department of War

over whether it can continue to be used in it.

Because whether or not you’re.

Can you describe what is happening there?

I can’t talk about discussions with an extremely important

partner that are ongoing.

So I’ll just have to stop it there.

So well I will describe that there is some dispute,

I guess my question, because I recognize you’re not going

to talk about what’s going on with you and your partner,

but it’s about a broader issue here,

which is there is going to be a lot of offensive possibility

in advanced A.I. systems, and one of the strongest drivers

of the speed at which we’re going with A.I. is competition

with China.

Some of the biggest risks that we think about

in the near term are cybersecurity

or biological warfare, are all kinds of ways

that others could use these against us, our drone swarms.

And there’s going to be a lot of money in this and a lot

of players in it, and it really seems unclear to me how

you keep this kind of competition from spinning

into something very dangerous.

So without talking about what you may or may not

do with the Defense Department, how has

Anthropic thought about this question more broadly?

We’ve been long term partners to the national security

community, and we were the first to deploy on classified

networks.

But the reason for that was actually

a project which I stewarded, which

was to figure out if our A.I. systems knew

how to build nuclear weapons.

This is an area of bipartisan agreement where people agree

that we shouldn’t deploy AI systems into the world that

know how to build nukes.

And so we partnered with parts of the government

to do that analysis that maybe illustrates what I think

of as for a thing to shoot for not just us,

but all the A.I. companies is how

do we both prevent the potential

for national security harm coming to the public

or proliferating out of these systems?

But also the second part is, how do we just

improve the defensive posture of the world?

And I’ll give you an example that I think is in front of us

right now.

We recently published a blog, and other companies

have done similar work on how we

fixed a load of cybersecurity vulnerabilities

and popular open source software using our systems,

and many others have done the same.

So yes, there will be all kinds of offensive uses

and there will be societal conversations

to be had about that.

But we can just generally improve the defensive posture

and resilience of pretty much every digital system

on the planet today.

And I think that will actually do a huge amount

to make the whole international system more

stable and also create a greater defensive posture

for countries, which helps them feel more relaxed

and relaxed.

Countries are less likely to do

erratic, frightening things. That

would be good if it happened.

My worry is, as an individual that I feel the opposite might

be happening.

So I’ve just watched people installing all kinds of fly

by night A.I. software and giving it a lot of access

to their computers without any knowledge of what

the vulnerabilities are.

Yep. I myself am nervous about using things like Claude Code

because I am bad at talking to Claude Code,

and I don’t understand these questions,

and I’m worried about loading onto my computer or something

that is creating security vulnerabilities I don’t even

understand.

The number of just scam voice messages I get every day.

Everything that are clearly somewhat A.I. generated,

or many of them seem to me, is very high.

There’s a question of societally,

do we use it to upgrade our systems?

I’m actually curious for your thoughts individually,

because as we’re all experimenting with something

we don’t understand and giving it access to the terminal

level of our computers without any real knowledge of how

to use that, it seems like we might be opening up a lot

of vulnerability all at once.

It’s the early days of the internet all over again,

where there are all kinds of banners for different

websites, or you could download like MP3s

to your computer that would completely break your computer

or download like helper software for your Internet

Explorer taskbar.

That was just like a phishing device.

We’re there.

We’re there with A.I.

We’ll move beyond this, but I believe that people,

when they experiment, come up with amazing, amazing,

useful things as well.

So my take is you have to say, when you’re doing the thing

that might be extremely dangerous and put big banners,

but mostly you still want to empower people to be able

to do that experiment.

So when you look forward, not five years,

because I think that’s hard to do, but one year, yeah,

we’ve kind of pushed into agents fairly fast.

We push into code.

I think a lot of people think code might be different than

other things, because it’s a more contained environment,

and it’s easier to see what you’re doing has worked.

But from your perspective of being inside one of these

companies and also running a newsletter where you

obsessively track the developments of a million A.I.

systems I’ve never heard of week on, week on week.

What do you see coming now?

Like what feels to you like it is clearly on the horizon,

but we’re not quite prepared for it or won’t feel until

it’s arrived.

No one has.

Maybe the way I’d put it is sometimes and you’ve likely

had the same had the ability to have certain insights that

have come through of reading a vast,

vast amount of stuff from many different subjects and piecing

it together in my head and having that experience

of having a new idea and being creative.

I think we underestimate just how quickly

A.I. is going to be able to start doing that on an almost

daily basis.

For us, going and reading vast tracts of human knowledge,

synthesizing things, coming up with ideas,

telling us things about the world in real time that

are basically unknowable today.

But the amazing part is, people

are going to have the ability to know things that

are just wildly expensive or difficult to know today,

or would take you a team of people to do.

But the frightening part is, I think that knowledge is

the most raw form of power.

It’s intensely destabilizing to be in an environment where

suddenly everyone is like a mini CIA in terms

of their ability to gather information about the world.

They’ll do huge, amazing things with it.

But surely there are going to be like crises

that come about from this.

And I think for the actual mental load

of being a person interacting with these systems

is going to be quite strange.

I already find this where I’m like, am I.

Am I keeping up with the ability of these systems

to produce insights for me?

Like, how do I structure my life

so I can take advantage of it?

I’m very curious about how you think even having that ongoing

conversation with the systems changes you.

So let me I’ll say it from my perspective.

One thing I have noticed is that the Claude

is very, very, very smart.

It is smarter than most people who

know about a thing in any given thing.

That is my experience of it.

But it is not in the way that other people

are an independent entity that is

rooted in its own concerns and intuitions and differences.

What it is instead is a computer system

trying to adapt itself to what it thinks I want.

So as I’ve talked to it much more about issues in my life,

about issues in my work, various kind of intellectual

inquiries or reporting inquiries where I’m trying

to figure out questions that as of yet,

I’m at of early stage of exploration.