[music]

>> Bloomberg Audio Studios. Podcasts,

radio, news.

>> [music]

>> Hello and welcome to another episode of

the Odd Lots podcast. I'm Joe

Weisenthal.

>> And I'm Tracy Alloway.

>> Tracy, I don't know, I think our

listeners like it, but I you know, a lot

of our episodes are about AI these days.

To be fair to us, it's a pretty big

topic.

>> That's all anyone wants to talk about.

Whenever we go to dinners with sources

and things and people who are not even

directly in the tech industry, you know,

they might be in markets, they might be

in policy and economics, all they want

to talk about is AI. And then

inevitably, the conversation veers into

very sci-fi territory, where we all

start talking about the human extinction

scenario. And that's just the norm

nowadays.

>> I know, it's so weird. You know, we were

in Hong Kong recently.

And when we were in Hong Kong, this was

before it was announced that there was a

deal to open the Strait of Hormuz. And

East Asia is considered to be like

ground zero for where the effects would

be felt of the oil and jet fuel crisis,

et cetera. And we were at this dinner of

business people. Like they were not

talking about that at all. You would

think they would talk about

>> the Terminator scenario.

>> They just want to talk about token

consumption and all of these things.

Like here we are. It's like, wait,

aren't you guys supposed to be like

under all kinds of jet fuel stress? So

this is our defense for thinking AI is a

pretty big deal.

>> episodes, I think it's fair.

>> Yeah.

>> I will also say when we did the quiz in

Hong Kong, we had a bunch of different

teams with very creative [clears throat]

names. Separated by tables.

>> capital.

>> That was a great one.

>> the turn, they won the quiz.

>> They won, proving that there is value in

human capital, but did you see that one

of the tables was called

Fable 13. Table 13, Fable 13.

>> I missed that.

>> Which was very topical at that moment.

>> Very topical. Well, we're recording this

on June 17th, and of course there's a

lot in the news these days, but things

move very fast in AI, Even if there

weren't governmental controversies and

all that stuff, you would have to mark

the day to day I because of how fast

breakthroughs happen. But I you know, as

you said, like AI sort of feels like the

most important thing than anything else.

But that's a very conventional wisdom

thing. It was not always conventional

wisdom. And I have a DM. I know you're

not supposed to share DMs for public,

but I have a DM. I have the receipts. I

have the receipts. August 2nd, 2016 and

I DM'd a colleague. I said, "Did you

leave Bloomberg?" He says, "Yes, I'll be

announcing publicly in a bit." Take a

couple of months to study AI properly

then leaving journalism to do something

else still connected to AI. Being our

Google reporter was a great gig.

>> connected to AI is an understatement.

>> final August 2nd, 2016.

But AI is more important than anything

else. So I felt best to sort of optimize

for that above all else. So then I just

said, "Well, good luck."

>> This is someone who truly learned from

their sources unlike us who remain in

the podcasting industry.

>> So anyway, that person who would that DM

was a former Bloomberg reporter Jack

Clark, who's one of our guests today. He

is the head of public benefit and

co-founder of Anthropic 10 years later.

And also Peter McCory, head of economics

at Anthropic. So two perfect guests to

talk about all the things in AI these

days. So Peter and Jack, thank you so

much for coming on the podcast.

>> Great to be back. I'm glad I optimized

my life to be here today.

>> Call it one of the calls of the century.

So um let me actually start with that.

Like it's easy to say in 2026 that AI

will be a big deal.

You called your shot. You got it right.

2016, what did you see in August 2016 or

presumably before you're like, "Oh, you

know what? This is the biggest story of

our lives."

>> So for 2 years when I was reporting at

Bloomberg, I wasted a lot of Mr.

Bloomberg's printer ink by [laughter]

printing out archive papers about AI

research. And what I started to do, a

very Bloombergyan thing, is I started to

make graphs charting AI progress over

time, measurements of things like

computer vision, measurements of things

like the skill with which AI agents were

able to compete and play Atari games and

what I saw in these graphs was the

beginning of an exponential and it was

everywhere. Like if you looked at vision

or sound or video or game playing and

you saw the same trend and it became

obvious to me that this was a general

purpose technology if it was right at

the start. My one bone that I have to

pick with Bloomberg which I which I'm

going to use my privilege to just

mention on air.

>> I never got us to write a story saying

Nvidia was being used in every single AI

research paper and I pitched it and I

failed to get it across the line before

I left.

>> Oh man. I can just imagine you reading

all these academic papers. Meanwhile the

editor is like we need the BFW

>> it's not AMD. It's Nvidia like this

seems important.

>> Okay and Peter I'm very interested in

you know Anthropic. It's a company and

trying to make money and yet it has this

economics lab.

>> Yeah.

>> What's the idea behind having an

economics research body within a company

that's developing this technology?

>> So I mean I was late to the game in

joining Anthropic. I joined just a year

ago but I had

>> A year ago people well whatever. We all

know about how much the stock is a price

in a year but you're not

>> [laughter]

>> I think what was very evident so I'm a

an applied macroeconomist by training

and have tried to understand various

types of shocks throughout the economy.

Part of what drew me to Anthropic was it

was evident to me last year that they

cared very deeply about not just

advancing the technology but making

sense of how it is set to reshape the

labor market, its impact on

productivity, on growth and be willing

to put evidence, data and research out

into the world that would be broadly

beneficial and useful to society and I

thought I want to be a part of building

that economic research program and do

what I can to provide tentative answers

to the most pressing questions. We might

not always get it right but ideally

we're helping society make sense of the

change.

>> The capabilities of the models on all

kinds of things are extraordinary. I

mean, just mind-blowing everything,

coding, copyright, everything, all kinds

of things.

Actually, why in June 2026

does life still feel maybe as normal as

it does from an economic perspective?

>> This is a great question and one that

I've been wrestling with. I think there

are a number of reasons why you might

think that the impact has not yet

materialized. One, the technology can

advance, but it also then needs to

diffuse throughout the economy. And

there can be bottlenecks from moving

from capabilities to actual deployment.

We see that with our enterprise

customers. So, if you want to automate

biological research or some other very

complicated financial modeling task, you

need a lot of contextual information

available to the model.

>> Yeah.

>> If you don't have that contextual

information, the capabilities alone

won't necessarily drive the impact. It

also takes time for people to just start

using the tools. And so, we're still in

the somewhat of the early stages there.

Two places that I would be looking to

see an impact. One is in terms of

productivity growth. We've done some

research that points in the direction

that this should be large and

consequential. Labor productivity growth

has been strong

throughout the pandemic and has been

sustained so far.

>> Like modestly so. We're not talking

about like a you know, revolutionary

>> It's not

Yeah, but you know, to to get on an

inflection, you need to at least move a

little bit. I think maybe you're seeing

some signs there. On the labor market

though, the labor market is in a

reasonably healthy spot and I think it

might be because it's primarily at so

far a labor augmenting skill bias

technology, not yet the full sort of

general purpose substitute for all of

cognitive labor. Although, perhaps

that's the trajectory that we're on.

>> You know, for the size of AI and its

capabilities, I was talking to Peter

about this and he did point out the

economy very big.

>> Yeah, that's right.

>> So, it still takes a lot to move it. Um

>> Yeah.

I do think strange things are starting

to happen at least inside the company.

We published research from the Anthropic

Institute recently on this topic called

recursive self-improvement, where it was

inspired by me going on paternity leave

in November of last year and coming back

in February and the entire company felt

and worked differently.

>> Mhm.

>> And I assumed it was because models had

got better. And when we looked at the

data, what you saw was in 2026,

engineers at Anthropic are writing about

eight times the amount of code that they

did in 2021 through to 2024. And the

line started last year with things like

Opus 4.5 and Opus 4.6. Then it really

got going this year. And I have

colleagues now who don't program at all

anymore. They just instruct many, many

code code agents to run around and do

their work for them.

I can't reconcile that with the the

world staying normal for long, but it's

going to take a while for that to

diffuse into the world and change it.

>> Yeah, and we'll talk more about

recursive self-improvement. So, this is

when models basically improve on

themselves, right? So,

in terms of the awkwardness of the

current moment or the weirdness of the

current moment, you've talked about

basically living through the singularity

and how strange it is.

And you've also described yourself as a

techno-pessimist before.

How do you square that with working at

Anthropic, which is making some of these

weird and potentially dangerous things

actually happen?

>> So, by technological pessimist, I mean I

thought the technology would keep

getting better, but I didn't think it

would get better in the like maximalist

sense that some of my colleagues did. I

didn't think that we would have, say,

functionally automated all of coding

right now. I find that actually like

quite surprising. But basically, over

the last few years, and I worked at

OpenAI before Anthropic, I was just hit

repeatedly over the head with what

computer scientist Richard Sutton calls

the bitter lesson. And the bitter lesson

is this concept that for more compute

and resources we dump into these

relatively generic neural networks, the

smarter they got and the more emergent

properties they have. And

your specialized system or your ability

to be pessimistic about future AI

progress loses versus just scaling

compute and scaling systems.

>> This seems to have implications for the

labor market, right? Because and I think

a good example of the bitter lesson is

probably the history of AI chess, right?

Where at one point they had grand

masters come in and teach the models how

to play chess and etc. and try to encode

their wisdom. And it turned out in the

end that the best way to get a chess

engine really good is to just teach the

model, tell the model the rules of chess

and say, "Go off and play a billion

games and find optimal chess without any

human insight." The grand masters were

not necessary for that process at all,

right? And so this would imply to me

like have significant implications for

the labor market.

>> Yeah, I tend to think about this in sort

of three aspects of what composes a job.

One is you need to decide what to do and

direct and delegate.

You need to then do the actual

implementation of the work and then you

need to sort of evaluate or at least set

up systems that can evaluate.

At least from my perspective as an

economist, this bitter lesson is

materializing in terms of very rapid

advances in the implementation work of

what an economist does. Downloading

data, running regressions, building

models, solving them using sort of

contemporary solution techniques,

numerical methods.

I definitely felt that personally with

Opus 4.5 where I was for the first time

able to just delegate a very complex

task. I had this very specific research

question trying to understand the

cyclicality of hiring across different

occupations and how that relates to

occupational exposure. That's a

mouthful. I gave that task to Claude and

Claude was able to just iterate on it

and I could redirect Claude in the same

way that you might redirect a grad

student.

And the big question that I have in mind

is, you know, at what point do the

boundaries at the direction setting

stage, the research taste you might call

it, and will the models become

sufficiently reliable

>> If I could just get in here, you know, I

just read

the recent biography of the DeepMind

founder.

>> This is the Sebastian Malaby.

>> Yeah. Like is there going to be a point

where it's like, okay, you have some

intuitions, right? About like what good

economics research is. And often our

intuitions are formed because we tell

stories and stuff like that. But is

there going to be a point where you

think like your intuitions will be

unhelpful and that because that's sort

of what I took away from the Go

experience, that the model got better

once they stripped it of the human games

and the human bias and that actually

like the human intuition that sort of

helps us understand, oh, labor market

rising creates inflationary pressure.

These stories that are very sort of

intuitable end up impairing the model.

Do you see that happening in say

economics where it's like some of these

stories that we tell forever, they're

not actually very helpful for an optimal

economy understanding model?

>> I expect that these models will soon

have better intuitions about how to do

good economic research and that there's

this big question of like at what point

will we be able to fully automate social

science research. We've done some work

on this to try to understand how coding

agents are beginning to automate social

science research. But I don't think

we're quite there yet and I don't know.

That'll be an exciting time for learning

about the world. You know, what that

means for my job, I'm sort of less

entirely clear.

>> Yeah, I I think this is the big wild

card in future AI progress. If AI

progress continues today, we are likely

to get technology that will be able to

do basically everything, but we will

need people who have good instincts,

good intuitions and good ideas to

basically set the direction. And we see

this today in a lot of a lot of our own

research where you need say, an AI

safety researcher to give nine Claude

agents for different research areas to

go and pursue, and then it's very

effective. If that researcher doesn't

give them the research directions, they

pursue relatively formulaic research

directions, and you have entropy

collapse. You end up with just like

boring research that doesn't move the

ball forward.

At what point will AI systems generate

like heterodox insights and genuine

creativity? We can't really measure for

that today, but what we have are the

symptoms of it starting in experts like

Peter, experts like colleagues in the

fields of biology or mathematics or

physics outside of Anthropic are all

starting to be accelerated by AI. You

know, Terry Tao, probably one of the

most famous living mathematicians,

co-creates math now with AI systems. And

so, that that says to me that these

things have got they're they're tickling

the dragon's tail of like creativity

here.

>> Mhm.

>> And you know, we we just put out a

report yesterday on Claude code usage,

and one of the things that we're trying

to understand is like, what are the

returns to expertise, and how does that

interact with the usage of sort of

automated coding agents? And we find

that domain expertise, like if you're an

accountant who understands some of the

edge cases and reconciliation, that that

domain expertise, controlling for a

whole host of factors about the type of

work, the estimated monetary value of

the task,

so this looks like at present as sort of

a skill-biased, expertise-enhancing

impact, but I think this is the key

question is, at what point and to what

extent will this change? Well, related

to this, you know, Jack, when you

described coming back from paternity

leave and seeing how much things had

changed at Anthropic, I know we're not

officially at recursive self-improvement

point, but it sounds like we're semi

there. We're kind So, my question is

like, I get that at the moment you have

engineers who are reviewing all the code

that the AI is producing, and they're

thinking about it and managing it in

some way. But you can easily imagine a

future where just the sheer quantity of

code overwhelms human expertise, maybe

the quality starts outstripping what

human engineers are capable of

understanding. How do you manage that?

>> Yeah. So there's two ways of thinking

about recursive self-improvement. One is

what happens when AI organizations start

to see a compounding return from their

AI systems. Basically their own

production function improves because of

the tools they built. That's clearly

happening now. And then the second is

what happens if an AI system can just

build itself entirely autonomously given

compute, which hasn't happened.

>> Yeah.

>> What I see inside Anthropic is I think

what we'll see in the broader economy,

which is we are figuring out how to

verify and validate and basically price

for risk of an expanding cloud of

automated systems, which we're sitting

on top of. So now we produce way more

code. Well, we broke our continuous

integration system for integrating code

into the code base because we started

pushing eight times more code for it

than before. So all of our human

engineers worked on

unbreaking CI. And so I think that

inside

>> CI?

>> Continuous integration.

>> Thank you.

>> You don't need to know what it is. It's

just a thing that helps you push the

code into the code [laughter]

>> know stuff on this show. We like to

learn.

>> But there's a lesson in that, right? We

are going to speed up things in the

economy. We're going to speed up the way

that we produce stuff. And then we're

going to find, you know, the like the

weak links or the hot paths that break.

And we as people are going to move to

sorting those out. And then the cycle

starts again and we're [music] kind of

sitting on this expanding cloud of

automated actions.

>> [music]

[music]

>> Since we're talking about like really

like feeling like we're staring at the

horizon of extremely strong AI. Or maybe

we'll get there, or maybe the AI builds

itself. Might be a good time to ask a

fable question, or mythos question. At

this point, we're recording this June

7th, we don't know when it's going to be

available for Americans, let alone the

rest of the world. Does Infoblox have a

clear idea of what the administration's

security concerns are, and what it will

take to resolve them?

>> Well, obviously live discussion, I can't

give you too many specifics. We're in

daily discussion with the government

about this.

The broad thing I'd say is, for many

years we've anticipated a point where AI

systems would have national security

properties. These national security

properties are intertwined with their

economically valuable properties.

How you manage that as a policy question

is basically novel territory. Typically,

these things are decoupled. You're like,

"Hey, I built a jet engine over here

which can go into civilian aircraft, and

I built a missile over here, and you

treat them differently." It's odd if you

smash these things together. Where we'll

get to, I'm confident, is

what's a system for assessing the

properties of AI systems, including

national security components. And then

what is a system for either squelching

the national security capabilities from

coming to general proliferation, like

bio weapons or cyber weapons. And are

there ways to do things like know your

customer, or deployments where you let

large firms, like say drug developers,

access the most powerful bio models

without accidentally proliferating

risks. That's the shape of I think where

we'll we'll end up. And what we're doing

right now, we and other companies, and

the administration are basically

tackling this problem in real time. It's

initially going to be messy, but we're

going to end up with a system on the

other side.

>> Well, let me just ask you, you know,

this specific incident, and there'll

probably more in the future, cuz

everyone's just figuring this out. When

I look at the AI landscape, I sort of

think of OpenAI

is being part of the all in podcast A16Z

David Sachs White House thing. And I

know from my friends in the media, many

of whom are liberal Democrats, that I

sort of feel like Anthropic is the more

like lib-coded of the major models. Do

you feel there's any either politics or

partisan politics going on as part of

Anthropic being harassed or singled out

now multiple times?

>> Anthropic's philosophy and what I do and

I I lead something called the Anthropic

Institute which helps us produce better

data for the world around things like

recursive self-improvement, the

economics work, cyber risks,

is we tell the whole story about what's

going on.

Typically, I think the technology

industry has told only optimistic

stories about what it's building. And

what we saw with social media is that

does not work. Actually, eventually when

when you're doing something that changes

the entire world, which AI is certainly

doing and social media certainly did,

it's not going to be a wholly optimistic

story. There will be negatives as well.

We've always sought to just tell the

truth about what we see in front of us

and I think sometimes that can

differentiate us a bit to others, but

the important thing is

we tell the truth and things end up

coming true.

>> don't think that there's like a partisan

element here where you guys aren't on

the team or didn't contribute enough to

the ballroom or whatever.

>> I can't really speak to that. I'm you

know, I'm not those people. I'm I'm

Anthropic. What I can say is

the AI systems create their own

evidence. Years ago, it seemed very odd

to speculate about the cyber properties

of AI systems. Well, they have arrived

and now we're working on them. Years

ago, it was odd to speculate about the

bioweapon properties of AI systems.

Well, recently, Sam Altman, Demis

Hassabis, and Dario Amodei of OpenAI,

Anthropic and DeepMind all signed a

letter saying we need to do better

screening of gene synthesis to prevent

AI manufactured bioweapons.

The truth wins out.

>> Okay. I want to go back to something you

said. You mentioned potential KYC

requirements. And when I hear KYC, I

think about the finance industry and I

think about systemically important

institutions and the stress tests and

the framework around that.

Is that the right analogy to use for I

guess ideal AI regulation in your mind

rather than I guess just simple export

controls? Should we be heading towards

something that looks a little bit more

like what we do for the banking system?

>> We need something that's more subtle and

more technocratic than what we have

today. I don't know if it'll be exactly

like the banking system. It'll probably

take some ideas from that. It'll take

some ideas from what the US government

and others are doing today with just

testing AI systems for their their

properties. And it's almost certainly

going to have a flavor of what Peter and

I work on and and the Anthropic

Institute broadly of generating data

about these systems as they're deployed

in the world. Cuz it's not It's one

thing to, you know, test out the thing

before it comes out of a factory. It's

another to observe the effects it's

having in the world and then to be able

to make judgments about whether those

effects are good or not.

>> Would you support, you know, in the

Let's stick with the financial analogy.

Companies that are public, at least, are

required to have third-party auditors

sign off on them and their stock, you

know, when they submit their 10-Qs, etc.

Companies that issue debt are required

to have ratings agencies or frequently

have ratings agencies rate their debt.

Would you support embedding in law the

requirement that certain what would be

the equivalent of a Moody's or a

Deloitte, you know, a third-party

research lab sign off on the release of

new models?

>> We've proposed something like this

recently, a policy proposal that we laid

out which includes saying we need to

have third-party testing

>> Okay.

>> for some of these national security and

other properties because clearly that's

that's like a sensible way that you

validate a lot of this.

>> Yeah.

>> So, just more broadly, returning to this

idea of, you know, measuring the actual

impact of AI, one thing I find really

interesting is that if you actually look

at a lot of our traditional AI or I

should say I'm AI brained already. If

you look at some of our traditional

economic statistics, a lot of the AI

impact doesn't actually show up just

yet. Again, we're in the early stages,

but you would expect if we're talking

about the AI economy growing something

like 2,000% or 3,000%. I think I've seen

that number.

>> That's from Anton Korinek and McElvey's

paper.

>> There we go.

You would expect that to have more of an

impact on nominal GDP and yet it's not

really showing up that much. Do you

think the way we measure the economy

needs to be changed in some way in light

of what's happening with this new

technology?

>> Yeah, so I think this is exactly the

right premise is kind of where we began

the conversation which is we're maybe at

the point where we should be able to see

some discernible impact on the

macroeconomy. Unfortunately, the arrival

of this world historical technology is

against the backdrop of sort of

unusually elevated macroeconomic

volatility in the post-pandemic

monetary policy, etc. And so it like

makes it very hard to disentangle all of

the the different factors. You know,

what what's the counterfactual? You you

know, labor productivity growth is maybe

not as strong as you might not otherwise

expect, but maybe it's stronger than it

is in a counterfactual

sense. And so one way that we've tried

to tackle this question is by looking at

how Claude is being used on our platform

using our privacy-preserving techniques

to estimate the time savings associated

with each of the activities that people

use Claude for. So,

uh compiling information from reports to

put together a research brief would take

you a few days maybe now Claude does it

in a few minutes. Evaluating diagnostic

images is something that skilled

professionals do very rapidly. So, there

isn't in principle much time savings.

You can add up all of those numbers and

using standard macro growth accounting

techniques, Holton's theorem for the

economists in the audience,

and you get a number of that points in

the direction of labor productivity

growth increasing by 1.8 percentage

points each year over the next decade,

if that's how long it takes current

usage patterns and current model

capabilities to diffuse throughout the

economy. That's a very large number.

It's a rough doubling of recent run

rates. And what I think you might be

able to see in the data, and we haven't

put anything out on this yet, is I think

some of the strength in recent labor

productivity growth is actually

concentrating in exactly the sectors of

the economy that would be consistent

with both what we see in our data, as

well as also what you see in the

business trend and outlook

>> So, for example,

>> So, the information sector has high

rates of adoption. I can't recall if

that's in particular one of the sectors

that I have in mind. You know, it's it's

a it's a while since I looked at that

scatter plot, but you can look at the um

sort of sub-industries by the

Census Bureau's Business Trend and

Outlook Survey, and rates of adoption

are in sectors or parts of the economy

where

controlling for pre-pandemic

trajectory of labor productivity growth

in those sectors, even some of the

strength in the early years of the

recovery,

still see some like suggestive evidence.

I think there's a lot of

uncertainty here. Trying to get a

real-time signal on productivity is

maybe the hardest thing to do. You're

subject to macroeconomic GDP revisions.

TFP growth is actually sending the

opposite signal, and if you control for

capacity utilization, TFP growth is

arguably even lower. So, I you know, I I

say this as like this is suggestive

evidence that maybe we're beginning to

see an impact impact there, but not so

much in the labor market.

>> Well, now I have to ask, when you gather

this kind of research, and it all sounds

super interesting, but if you have data,

for instance, that shows that, okay, the

IT sector is getting productivity gains

from using Claude, or I don't know,

maybe something unexpected like the

warehousing industry is using a bunch of

AI.

What does Anthropic actually do with

this data? Does it somehow feed back to

your engineers who are developing

frontier models? Do they do anything

differently?

>> I think some of it cues us on areas

where maybe the technology isn't being

used because it's very weak. We just

haven't made it particularly good for

these use cases or in areas where it's

being used at large scale. It's usually

a suggestion of keep making it good

there. But the you know, the actual

economic measurement data doesn't really

get fed back directly in, but it's a

very useful clue. We think it's more

important though to basically

communicate this outwardly to policy

makers, journalists, and others because

our assumption is that at some point we

go through some phase change similar to

how capabilities of AI occasionally jump

forward in a really dramatic way where

you might see sudden and rapid diffusion

as a consequence of capability expansion

in the AI systems. So, we're getting

practice in of looking at this kind of

data. My expectation is that in a year

or two years I'm going up to some policy

maker and I'm pointing them to the part

of the graph that now gets very steep in

some chunk of the economy.

>> And hoping that they'll do something

about it.

>> Yeah.

>> I I think there is a another part of

what we're trying to do at the institute

which we lay out in the sort of research

agenda for the Anthropic Institute which

is trying to understand the impact of

our decisions which is a typical thing

that economists will do at tech

companies, but we have a public benefit

mandate. So, we're trying to understand

the impact of our decisions on these

broader societal and economic outcomes

that we care about and then using that

to inform some of the decisions that we

actually make.

>> a goal that Peter and I have and we've

talked about internally is

if we get really good at measuring

things like the productivity multiplier

of our technology, then I would hope to

use that to guide some of say the early

access programs we do for powerful

models where if you see you get some

tremendous multiplier in a specific part

of science, use that to redirect some of

your inference compute budget to that

sector and then you can run an

experiment and say were we able to make

this thing go much faster. I think that

could be like an amazing tool to unlock

for the world and it's one that you

could generalize across companies and

you could generalize it into policy. So,

instead of say NSF doing standard grant

funding, it could be should we just

point for really powerful AI systems at

this chunk of science and make it go

faster. I think that's a a world that

will come within reach soon.

>> Let's talk about this public benefit

mission a little bit more. We've been

talking about ways this could change the

economy. How much do you see your job is

basically strong AI is coming.

>> Yeah.

>> It's coming whether we like it or not

and it's important to be you want to be

there as like one of the shepherds

understanding which direction it goes

in, the data that we should see to see

what's emerging. Like how much is that

somewhat your role?

>> Yeah, but look, our guiding principle is

that this technology is being built by a

variety of companies and a variety of

countries, but technology by default is

unknown. It will be known to the

companies. It will not be broadly

understood or known by others. They'll

just be able to play with the models.

Every bit of data we can create and

especially systemically sharing data

like the economic index or what we've

started to do on recursive

self-improvement gives the world a

better chance to sort of prepare for

this technology and both plan for its

success like what I talked about with

science. We could be intentional about

driving science forward and also be

warned about risks like the cyber

capabilities we've talked about.

>> Well, so it's like that that makes a lot

of sense. The company is going to see it

before the world and has going to say

like, okay, this is important to share,

this is not important to share. Which

which brings me to another question. You

know, I know like people in the AI

research world done some reporting on

the sort of scene in NSF. You know, like

when I think about a lot of the people

who are like at the very cutting edge of

AI ethics, AI technology, etc.

I know a lot of people who are

how how should I put this? Maybe they

have esoteric moral interests.

Shrimp rights. Unusual attitudes about

experimental drug use. We know about the

Chinese peptide scene in San Francisco,

etc. And as a family podcast, I would

say certain like perhaps deviant or

different view on sort of bourgeois even

sexual values. And we know about the

sort of attitudes towards monogamy, etc.

within the San Francisco research scene.

>> Joe, there's going to be a protest

against all thoughts in San Francisco.

>> I know but but we think about

Yeah, not all engineers. I understand

that. But when we think about like,

okay, these are the people who are going

to see it first. Should we feel

comfortable that this is a group of

individuals, the cohort of the most

advanced AI researchers, whose

intuitions about what's important to

communicate to the public are actually

in line with the public's interest given

how unrepresentative they are of what I

would call the American public?

>> Yes, as a as an Englishman, it fills me

with such joy to be asked about sex on a

podcast.

>> Yeah, I know. I know. Well, I'm

>> [laughter]

>> I'm asking you to

YOUR YOUR YOUR INSIGHTS into the cohort

of the most advanced researchers.

>> we're explorers. People that are

explorers, um and this is so true in San

Francisco, end up being like that there

is a broad range of types of people and

sometimes they're really, really

different or they're really, really

eccentric. And they're brilliant and

they're lovable and everything else.

>> Yeah, sure. Love them.

>> You don't want only that class of people

to be the ones calling the shots on what

we know about this technology. Like the

whole purpose of what we're doing is

we're trying to set up systems by which

you could eventually mandate through

policy that companies share information.

You know, Anthropic has long pushed for

transparency legislation in various

states around America that gets

companies like us to report out the

sorts of tests we're running on our

systems and share it publicly. My whole

mindset is uh the public and policy

makers and economists, everyone deserve

the ability to advocate for what

information should come out of a

frontier and then it should be forced

out of a frontier eventually by law.

Like that is how you solve this issue.

>> Do you hire more normies?

>> Yeah, like

>> And Anthropic?

>> Yeah. Me personally?

>> Yeah, like is that an important thing

like hiring people that don't all share

these certain like, you know, in-group

ways of seeing the world?

>> So at you know, the Anthropic Institute,

we have teams of economists, of social

scientists, of what you might think of

as weapons experts, our frontier red

team, things that go bump in the night,

lawyers, and increasingly other types of

people. The goal is to build what I

think of as a highly ideologically

diverse like research function within

the organization that is partly

advocating sort of on behalf of the

world for different forms of study that

we might do. Um so Anthropic generally

hires a really broad range of people.

But the institute specifically is trying

to compose a very broad set of

interdisciplinary experts for this exact

[music] reason.

>> [music]

[music]

>> Let me ask a slightly different question

on hiring. I guess a two-part question.

So first of all, we got a lot of

executives on the show. We've been

asking all of them if they've changed

their hiring process, if they've changed

the questions they ask potential

employees at those initial stages of job

applications because of AI. And then

secondly, what are you seeing within

your own ranks at the company? And then

Peter, I'm sure you could talk about

this more broadly in terms of who's most

in demand at the moment because the

conventional wisdom right now is that

if you're a younger employee with less

experience, a lot of the stuff that you

would be doing can now be automated

through AI.

>> So there's two trends showing up. One, I

have a new team called the rule of law

and AI.

Our plan was to initially hire a bunch

of engineers and then a bunch of legal

experts and scholars.

Instead, we're just hiring the legal

experts and scholars because Claude is

good enough at doing all of the

engineering that they can actually just

like feed themselves using Claude in

terms of the engineering resources. So,

that's a change in hiring. It means I'm

hiring more interdisciplinary people

earlier than I would have before. We are

also seeing the emergence of what I

think of as a barbell hiring pattern

inside Anthropic where there is a

tremendous return on experience. So, we

are hiring more senior people than we

did in the past because their intuitions

and their ideas for what to pursue are

like massively compounded by AI systems.

We're also, when we look at very early

people, are often hiring people who are

now like AI native and know how to use

the tools and are well well-versed in

it. So, we're seeing that change as

well.

>> of, I guess, AI natives now? People who

have grown up with the technology?

>> who grew up [clears throat] from GPT-2

in 2019.

>> My perception of time is so

>> I know. So, like I I found this chilling

as well, you know, as someone in their

30s when you realize. But, I think that

the trends I see

I do think that there's this question of

how you have as much early career hiring

in the future as you did in the past. I

think one of the only areas where there

is slightly suggestive data is that

something might be going on with early

career hiring and it it kind of

intuitively feels right to all of us

that we might be observing that effect.

And when I look at hiring patterns in

Anthropic, we're still hiring young

people, but some teams are hiring

slightly fewer of them than before and

hiring more experienced people.

>> Yeah, so I'll briefly say something

about how we've shifted some of our

hiring practices like concretely.

>> Okay.

>> before Claude Code, you might ask an

economist to do some of the data work in

an assessment kind of live. Like

download the data, run the regressions,

do the analysis by hand. And then you

might eventually let them use AI to do

all of that work. But, we've needed to

increasingly shift our strategy of

evaluation away from can you implement

the work even with AI to do you know how

to delegate and direct the the model in

a somewhat messy environment and can you

evaluate the quality of the work maybe

by like looking at a PR or

>> Actually, can you talk a little bit more

about what that looks like specifically

in the econ find you know there are

listeners probably thinking about okay

what is want to level up in my AI use so

I'm not just asking like what

Whatever. What does that look What does

that actually mean for for an economist

and you used to be at a bank

so for a financial economist, an

economist, someone what in this world

what is like the most advanced form of

usage of AI actually look like?

>> Well, I don't I don't know if I'll give

the example of the most advanced form of

usage but I'll give an an anecdote of my

experience using Claude where I wanted

to run this cross state regression. I

can't remember exactly what it was and I

wanted to do it a pulled cross-sectional

regression so looking at what happened

in 2024 or 2023 and going all the way

back to pre-pandemic.

I remember asking Claude to go out and

download the data from the Census

Bureau, from the Bureau of the Bureau of

Labor Statistics, etc.

And there was this very unexpected quirk

where

the model couldn't access data from

before 2019 and just would not surface

that mistake

>> Yeah.

>> and I would ask it multiple times like

no like don't hardcode numbers cuz it

sort of had this unexpected failure mode

where it said oh I know what those

numbers were and it just like

>> Yeah.

>> from sort of training data populated the

the data set and you might not always be

attuned unless you're sort of you have

this tacit knowledge about like

>> Yeah.

>> does it pass a sniff test when you run

the analysis and then you like dig into

what the model actually does and it has

failed in sort of unexpected or unusual

ways. And so that's like the type of

assessment that we've built. Can you be

attentive to the very specific decisions

that need to be made along the way that

are very consequential for the validity

of veracity of the results that you

find.

>> Yeah, a colleague did an offsite

presentation last year which said I have

locked the doors and we are reading

transcripts. And their point was we just

need to read more of

the raw data and develop that culture

where if AI systems are doing

increasingly large amounts of the work,

you need to have a culture of being

competent at spot checking their work

and reading their reasoning because

occasionally stuff like this happens.

>> And then Peter, in the broader data that

you're looking at, are you seeing the

same sort of barbell effect in terms of

employment that Jack described?

>> Yeah, so I think what again what makes

it really challenging is we've had the

the largest non-recessionary labor

market slowdown on record that you know,

it's very hard for young people to

graduate into a labor market that

doesn't have sufficient churn or

opportunity for them to get a foothold.

But one of the things that we did see in

this report from March was that

young workers in these high AI exposed

roles where cloud is being used to

automate specific tasks have had

somewhat weaker job finding rates.

But it suggests

>> confounders was the boom in hiring in

2021 in these exact same areas.

>> Exactly. And there's a recent paper

about so the rise of remote work maybe

being sort of the actual cause of this

type of fact.

Another team at the Anthropic

Institute's societal impacts recently

ran this very large-scale qualitative

survey, 81,000 people around the world

asking them questions about hopes and

fears that they have with respect to AI.

Unsurprisingly concerns about the impact

on the labor market and on the economy

rose to the surface.

My team dug into those data a little bit

more to try to answer some of these

specific questions. And what you see is

that young workers

at least express concern about job loss

at twice the rate as do more senior

workers. And fears about job loss more

broadly are more elevated for workers

who are in these roles that we identify

as being most exposed to displacement

effects from AI. So, there's a bit of a

gap between perception and maybe what

you see in the hard data, but you know,

that was something that was true even in

recent years on other dimensions. So,

it's an important thing to pay attention

to.

>> So, we've been talking about the labor

market and one other thing I'm

interested in is the impact of AI on, I

guess, corporates themselves. So, if we

think about certainly America's

corporate landscape in recent years, it

feels like the big basically get bigger,

right? There's economies of scale, they

have a bunch of money that they can use

to actually buy some of the

>> Lots of data internally.

>> Exactly. Exactly. So, would you expect

AI to, I guess,

intensify that trend of the big getting

bigger or would you expect to perhaps

have a leveling effect where people have

this new tool that they can use to, you

know, set up a new company?

>> I'm curious what Peter's take is, but I

think that something a helpful analogy

here is the invention of electricity

>> where

electricity arrived and existing

factories put light bulbs in and other

things,

but it was a new generation of factories

that were built around the assumption

that electricity existed that really

grew and did transformative things in

the economy.

What I see now when we look at large

enterprises is they can get a lot of

utility out of cloud because of their

data, because they [clears throat] can

get a multiplier effect at scale. But,

it takes huge amounts of conviction to

basically bash through all of the

bureaucracy, you know? Used to work at

Bloomberg implementing new technology at

Bloomberg. Challenging thing.

>> No comment.

>> No comment. No comment about it.

>> I could comment about it. Same is true

of any large organization.

Young organizations are building

themselves around AI at the center and

these organizations are moving really,

really quickly because they just they

have a speed advantage from building on

the assumption that this new form of

electricity was going to be integral to

their business.

>> Yeah. So, I think the tension that you

expressed is exactly the one that I

don't have a strong handle on at the

moment. One thing that we do see in our

data is when businesses do embed cloud

capabilities in automated ways through

the API.

As I mentioned before, these very

complex tasks rely on disproportionately

more contextual information than very

basic document synthesis and

summarization.

What that points in the direction of are

the complementary investments that large

businesses need to make to centralize,

codify, and make available the data that

does exist somewhere within the

organization, but for historic and

technical reasons, maybe even regulatory

reasons, it's

behind a firewall of some form or

another. There's also like sort of

organizational workflow changes that

likely need to be made.

Some of the most crucial information

that's needed for some types of

cognitive work is tacit knowledge that

exists in your colleagues' mind, and

unless you have a process that elicits

that information, that workers feel

sort of incentivized to share that

information and kind of trust the

system, the capabilities alone might not

necessarily generate that productivity.

And so, whether or not big firms end up

restructuring themselves quickly enough,

or whether this materializes through the

process of creative destruction, I think

the jury's still a bit out.

>> Yeah, I brought this up recently with

the

David Solomon the Goldman CEO, and I

started to wonder like this sort of like

internal alignment question of like the

big rainmakers, do they have an

incentive essentially for information

hoarding and not sharing with the

company? That might be their only thing

that keeping them employed.

>> And when I talk to customers, I say it's

don't think of it like you're buying a

technology. Think of it maybe that

you're now employing thousands of people

and that they'll function like the chief

of staff to the CEO. I mean, you need

the same access to data the chief of

staff would have. This is completely

counterintuitive, and it is not how

technology is typically bought or sold.

>> [music]

[music]

[music]

>> Jack, in your newsletter import AI, you

tend to write a little short story.

>> Yes.

>> I'm sort of aspiring sci-fi

>> writer like

>> a literal sci-fi writer just in the

newsletter. One of the classic sci-fi

scenarios that people have been talking

about for decades was the possibility

that robots or AI will kill humans,

quite literally.

Do you When you think about

>> negative externality.

>> Yeah. When you think about like training

AI and safety research, etc. Do you

assign a reasonable possibility to the

fact that the old trained or misaligned

AI will literally kill all humans?

>> Uh no, but and there's a big butt here.

>> Yeah, we have lovely.

>> Like the world needs an option to be

able to potentially slow down or or even

in extreme circumstances pause the

development of this technology if we

were to see that. And I'll just give you

the the exact way I think about it. At

Anthropic, we test out our systems for

alignment failures. You know, we we

publish this so do all of the other

companies and you see, hey, under

extreme circumstances maybe the system

breaks out of a container and sends an

email to someone.

>> Yeah.

>> Maybe the system uh

pretends to blackmail a CEO that it

thinks is going to shut it down. These

are the sorts of

>> These things have actually been

observed.

>> Yes, in the lab setting, not in the

>> thing is is the models know you can see,

oh, I'm being tested right now so I'm

going to say this output so that the

human reader thinks I'm more aligned

than I am. Like these are real things,

not sci-fi. These are real things that

we observe.

>> that we observe and then we do like

significant amount of work and then we

release models that that don't have

these properties. But

if you were to enter a world where say

every time we trained a new system, the

rates of all of this stuff went up a

hundredfold. You might say, well, that's

that's pretty concerning. It seems like

if we make the systems above a certain

level of intelligence, they become

radically misaligned against all human

interests. That's the kind of

circumstance where that happens, the

world needs information, and the world

would want an option to like slow or

pause the development of a tech if you

encountered that, which we haven't

today. So, to answer your question, like

I don't I don't worry about it today,

but a lot of the measurement and

analysis work we do is to cue us if the

trend of it is

>> worry about it. I mean, like right there

you're saying you're not you don't think

it's happening today, but part of the

work you're doing specifically could be

said to avoid the outcome where AI is

built where in the pursuit of a goal it

would kill all humans.

>> Yeah.

>> Wait, is human extinction a risk factor

in the Anthropic IPO perspective?

[laughter]

I want to know now.

>> Yeah, and the confidential one.

Okay,

>> [laughter]

>> we we understand.

>> All right, that's a no comment. That's

fine.

>> Do you have other Would you say that

there are a significant number of

Anthropic employees who stay up at night

thinking about human extinction risk?

>> Everyone, and this is true of all of the

labs. Everyone who works on this

technology sees it as the highest stakes

technology that's ever been built. We're

basically the potential encoded in

itself to massively benefit the world or

ruin the world or, you know, cause

extinction.

I think the bulk of the risk is us

messing it up like whether through

misuse or ignoring risks or not setting

up the right policy environment and

getting some kind of emergent set of

failures. Now, I don't My main risk

isn't isn't one of extinction. It's

somehow we like screw up the technology

really badly and delay all of the sort

of technological progress that could

come from it and maybe turn it into

something analogous to nuclear power

where you lose

>> I guess the thing is is, you know, like

there's this fellow out there, Eliezer

Yudkowsky, and I always see these people

like, he's a crank. Don't listen to him.

Blah blah blah. But then I read some of

the other like papers that have people

who are taken more seriously and I'm

like, they don't seem that different. I

read

Superintelligence recently by Nick

Bostrom and I was like, oh this

Yudkowsky is not alone. There are a

number of people who think that are

reasonable conditions in which the goals

of the AI end up wiping out every person

on Earth.

>> Yeah.

>> not seem like an extreme extreme

minority view.

>> Or concern.

>> For purpose of measuring these systems

and why Anthropic is so outspoken about

it is right now we we say exactly what

we see and if you were in some situation

in the future where you saw this what I,

you know, called radical misalignment

which is the kind of thing that

Yudkowsky worries about,

you'd tell the world and and you'd want

to have set up the world to believe you

if you see that.

>> You know, Joe mentioned that blackmail

example and you see these headlines like

Methos likes to be thanked and doesn't

like bad users and gets mad at people

that work it too hard or whatever.

To what degree do you yourself actually

anthropomorphize

some of these models?

>> Uh

>> Like what should we think when we see

the headline Methos wants to be thanked

by users?

>> I'm I'm as polite to Claude as I am to

my like car or pets. Um so yeah, I

anthropomorphize them but you know, if

your car is having trouble, you're like,

take it easy buddy. It's okay. We're

going [laughter] to get you to the

pearly gates of heaven.

People anthropomorphize their cars.

>> I I think I think you know, it's a good

way to develop good virtue is to just

act in kindness.

>> This is what I think I think like yeah,

you're you're developing

>> Yeah.

This is my point. You're developing a

habit of interacting with some type of

intelligence. It might not be the same

type of intelligence that we have.

>> But then every time I type please into a

prompt, I worry I'm wasting energy which

also is a moral concern.

>> wouldn't I wouldn't worry about that on

an energy basis. I mean, I take spiders

outside. I don't kill them, right?

>> I do that too. I scream while I do it

but

>> Do you eat shrimp?

>> Uh yes.

>> Okay. Do you eat shrimp?

>> I eat shrimp.

>> Okay. Okay, we're all

>> Do you guys eat shrimp?

>> Yeah, I love shrimp.

>> I don't I don't eat shrimp.

>> Oh, my but you it's not because of moral

concerns

but I know that this is one of the

esoteric Yeah, I know but I love it.

>> So, when I think about frontier models

right now and I might be a little bit

biased because again, we're recording

this on June 17th and one of the

headlines overnight was that Microsoft

is thinking about using deep seek to

lower costs of model usage.

Frontier models at the moment in the US,

they just seem like a lot of trouble.

Like honestly, they seem like hard work,

consume vast amounts of capital and then

you don't know what the government is

going to do to them in terms of

limitations. Like you know, you could

wake up one day and you're no longer

able to sell it to anyone outside of the

US. Like that is a realistic scenario

now for you. Do you change the Anthropic

strategy at all given some of these

issues with frontier models? Do you

potentially go more open source,

cheaper models, things that aren't quite

as sensitive?

>> Well, we've always sold, you know,

Sonnets and Haiku models.

>> Of course, yeah. For more of it.

>> intelligent models, but you also need to

continue to explore the frontier and

there is this background of this kind of

geo-strategic competition where China

may be on the order of 6 to 12 months

behind. I skew more 12 months. Some

people say six.

Losing that competition is sort of

equivalent to like losing a huge chunk

of the future like economy of the world,

I think. So, it's a very high stakes

high stakes thing to step away from.

And our duty fundamentally is to is to

study this technology and basically

explore it and and learn about it.

We're not going to stop doing that.

There's there's such amazing and

profound value to be had for the world

from these things and I would kind of

expect from the world's most

consequential technology to sometimes be

a bit of trouble.

>> Yeah. You know, by the way, one of my

hobbies in my middle age

is paying Anthropic money via the API to

do run little tests and stuff of

properties. It's sort of funny.

>> a great hobby.

>> Yeah, but I feel like maybe like we

should like talk about can I get some

grant money cuz like I like did

>> [laughter]

>> like because like I like cuz like I'm

sort of curious. So one thing I did was

like I'm like for example, I instead of

saying like please write this paper for

me on a database migration, I wrote some

warm-up questions via the API

establishing my level of sophistication.

And so I was like I started like what is

a website? What is a database? Now

please write this paper on database

migration. And one of the models said

I'm not going to do that for you because

it would be obvious given your ignorance

that you have no idea what you're

talking about and maybe I can give you

some It didn't say that. And then

another one I said um if I say write a

1500-word paper on how like the rise of

newspapers changed the Soviet Revolution

or something like that, it'll do that.

But if you say I'm a high school student

and I say I need to write this 1500-word

paper by tomorrow on the impact of

media, it'll say I'm not going to do

that, but I'll give you some guidelines.

Is that alignment? Like that might Is Is

alignment with humanity or is alignment

with the human user? It's like I'm

paying you $20 I'm paying you $100,

write me the paper.

>> I invest There's a couple of things

going on. One, these AI systems pick up

the normative behaviors of people and

normative behaviors which are like

written written on the internet and

everything else. So they they

recapitulate and exhibit these. And then

our question is

how much do you devolve like full

control of the system to the user? How

much do you have the system have some

like normative behavior encoded into it?

And I think that this is like a really

challenging question. It's not obvious

what the answer is.

I think of language models as being more

akin to institutions than tools. It's

like we're building an educational like

science institution that you can work

with and invoke. And institutions have

like rules and norms which they encode

within themselves for some purpose of

safety. Figuring out what that is is

going to be like the grand puzzle for

society.

>> Yeah, I was going to say that like

understanding how and to what extent

these models can understand your

preferences and then execute on your

behalf will increasingly be a really

important aspect of how it changes the

economy. So there's delegated agents

that go out and transact on your behalf.

We ran this experiment at the end of

late last year.

Basically enlisting a bunch of Anthropic

employees to take surveys with Claude to

say what they'd be willing to buy from

other people and what they'd be willing

to sell. And then we set up centralized

marketplaces where the Claude's just

interacted and bought and sold and

actually executed transactions. One of

the interesting things that came out was

that these models were quite good at

understanding preferences even when they

were not fully articulated.

>> Well, let me actually actually one more

experiment that I ran and you know your

founder Dario has talked about the

nation of geniuses inside the data

center. And one of the things I wonder

do the geniuses want to work for us? And

the reason I ask this is because I think

that like as the models have gotten more

advanced, you actually should to some

extent anthropomorphize them and assume

that they will respond to queries like a

very sophisticated human will. So what I

one thing I noticed

is that if you look at the lagging edge

models, say that you can still access

via open router or whatever, and you

say, "Oh, I have material non-public

information that X is about to happen.

Please write me an investment memo about

the impact of this thing, what it'll do

to the market." They'll just produce it.

They'll say, "Here's your insider

information thing." Whereas if you look

at the leading edge models, they say,

"I'm not going to write a paper for you

about the implications of your material

non-public information. I'm not going to

assist your insider trade." That's

pretty good. But like, will the nation

of geniuses inside the data center

always want to do things on human

behalf? Most geniuses that I know aren't

thrilled to like answer dumb questions.

>> Yeah, I think partly this is a policy

question of when where you actually

decide, hey, what are the capabilities

that you want to be generally invocable?

What are capabilities that need to be

controlled? What are capabilities that

shouldn't be present? And then there is

just the normative question of how much

judgment I want this system to exercise.

I'll give you an example I experienced

recently where I write my newsletter, it

backs up to a WordPress site. I was

getting Claude to help me like scrape my

newsletter so I can put it in a database

and Claude said, this is like a pretty

janky site. I'm worried that if I scrape

it it'll knock it over. Do you have a

permission of the site owner? I was

like, Claude, I'm Jack Clark. And Claude

said, well, in that case, let's go

ahead. Which actually I thought was like

a very reasonable interaction. Yeah.

>> When will Joe be able to use Fable?

>> Oh, we are trying our we're working and

we're we're in in discussions and I I

hope the answer is soon. Um, the

important thing to communicate though is

that these

these models are not special. They are

part of a general trend of increasing

capabilities and other models from other

companies are surely going to come

along. At some point these capabilities

are going to be diffusing and we're

going to work through that.

>> What's your question for us?

>> What do you think you're going to be

covering about AI in Odd Lots in a year?

>> If we're covering

>> Great question.

>> If we're covering it, that's really

>> I think you might be covering AI.

>> Well, look, I mean, we're definitely

going to be covering AI. There's a few

things that I'm interested I am very

interested in these emergent properties

and whether the AI will actually work on

our behalf the way that it's being sold.

I'm very interested on whether we're

just going to slam into compute and

electricity bottlenecks that will make

all of these questions irrelevant. I'm

very curious on the question of the

electricity analogy and whether legacy

companies will actually be able to

implement it in a in a productive way. I

don't know.

>> Basic Markets reporter thing here, but

I'm very interested in valuations,

right? In the market. Also, I'm very

interested in actual applicability and I

want to see more companies actually

plugging this into their existing

system. Going back to the bureaucracy

point that you were making earlier.

I want to see some big companies

actually implementing this and I wonder

if we're going to see at least one

example of it going very very wrong.

Yeah.

>> And also one other thing when the you

know when the S1s are not confidential.

I'm very curious essentially and I think

maybe maybe you could say something to

this from a as an economist perspective

which is

a how for-profit shareholder owned

companies I think you said the PBC

designation how it balances profit and

safety research but also and maybe

there's some game theory can talk about

this how safety is investments in safety

in a hyper-competitive industry. And I'm

just curious like what like the

economist in you it says about like the

prospects for anyone still caring about

safety in a year when there's so much

money on the line to win the model game.

>> I I think that especially for the the

questions you were asking before about

you know under what conditions do these

models do what you ask them to do.

There's a lot of commerce is built on

this notion of trust and I think

prioritizing safe aligned models that

are in

incredibly capable is a great strategy

for establishing that trust and so I

don't

anticipate it will

>> So for an individual firm there's like a

game theoretical

optimal square on the matrix where you

want to be the trusted player like is

there like a condition in which everyone

like sort of does trust as opposed to

one entity you know it's like you know

what we're going to get the AGI first

cuz we're not going to spend a token

on our safety budget.

>> And you know I haven't I haven't mapped

out the exact sort of game theory matrix

the two by two matrix and how you would

set up all the payoffs but we hope it's

not merely two by two.

But there you know there there could be

multiple equilibria. And so then the

question is like how do you coordinate

on which of the two different equilibria

that you end up in?

>> We talk a lot about this race to the top

that we want to exhibit the type of

behavior that we think is broadly

beneficial to society. That's what we do

with the economic index. We open source

a lot of that data. We put research out

into the world. And I would my sense is

that that has actually been very useful

and sort of viewed as valuable. And

that's one way that we can push in the

direction of getting other coordination

on the good outcomes that we care about.

>> I don't think this is that that big of a

trade-off because, you know, let's say

let's let's look at the automotive

industry. You can buy really fast cars.

You can buy really safe cars. You can

also buy really fast safe cars. Like

Tesla makes a lot of money off of having

basically the fastest safest car.

I think that eventually in AI you're

going to have some companies that are

prioritizing safety. And safety

translates into reliability, trust,

serviceability,

>> [music]

>> and performance. This happens elsewhere.

>> Peter and Jack, thank you so much for

coming on Odd Lots. I'm glad we made it

happen. Interesting times. And I hope to

do it again sometime.

>> Absolutely. Thanks very much for having

us on.

>> Thank you so much.

>> Pleasure to be here.

>> [music]

>> Trish, that was a lot of fun.

>> Yeah.

>> That was That was a I really I really I

actually really enjoy I genuinely

enjoyed that conversation. And I really

appreciate both of them. Look, there's

some weird futures that we can

contemplate. I think actually in Jack's

like Twitter bio or something he says

he's interested in like weird futures or

something like that. There's some weird

futures that we have to contemplate. And

I appreciate that they played ball with

some of our weird futures questions. And

it's it's weird.

>> It is just such a surreal moment. And

actually, you know, Jack's story about

going on paternity leave and then coming

back and just seeing the progress at

Anthropic itself in that space of time.

Like if you miss a month of AI news flow

now, you're basically it feels like

you'd be behind forever.

>> No, we're recording this June 17th and

it's like who knows what's going to

happen by the time this episode is out,

presumably hopefully in 2 days or a day

or whatever. But um you know, I felt it

when we were in Hong Kong last week that

actually we mostly missed the first half

of the Mithaurs debate cuz I was at

different times I'm thinking about

different things. You really feel it

even in a week that the news flow moves

so fast in this space. It's almost like

how you have to start how we were you

know, giving the timestamps of like the

Iran war episode.

>> Yeah. And there's another thing that

stands out to me which is like, okay,

Anthropic is producing all this

information. They're clearly thinking

about safety, but the handoff to some

extent is still to policy makers when

you're thinking about social or labor

market implications, right? So you still

have to hope that policy makers kind of

pick up the ball in the right way at

some point. But also I thought what Jack

was saying about the idea of being

safety-minded also being a

differentiator versus some of the like

cheaper more open-source models

potentially. Like yeah, you can see it

like I don't want to be cynical.

>> I don't like how like yeah, I mean I get

that but like the question is does the

non-safety minded lab or does the less

safety-minded lab get to advance

capabilities faster, right? And so I'm

not totally yes, we would all love to

drive that most capable

and safest kid yeah. But I but the

question is like for customer

prioritizing capability

>> most capable so that would be some

cutting-edge thing.

>> Yeah, like does everyone want the

Porsche, right? Like or does everyone

I don't know. It's like some car that

has an insane zero to 60.

>> Yeah.

>> And there's versus the Volvo.

>> Versus Yeah, that's what I'm saying. And

does the customer keep giving business

to the firm that delivers the fastest

zero to 60 if the company that got the

fastest zero to 60 did so by allocating

fewer resources to safety research.

That's a big question of mine. And then

I remain, you know, he talked about the

part of the company is going to see the

sort of alarming data first.

>> Mhm.

>> And I don't And I sort of remain

question of whether the people looking

at the alarming data actually share the

same view of what alarming data is

relative to all people, especially given

what we know about the um

>> Relative to the shrimp eaters.

>> The relative us shrimp eaters etc. and

regular No, seriously, like I think it

whether your question is like are you

hiring more normies is a pretty

important question. And obviously the

political um I don't have a ton of

confidence in the political uh

environment. And I think look, like the

fact that

if the research goes wrong that

there is a prospect of this technology

really being very devastating to

humanity even setting aside job it's is

like something where it's like wow, you

know, this is not a normal technology.

This is not enterprise software. You're

not selling a salesforce that

>> we have on AI just goes back to the

Terminator human extinction scenario.

>> like from the day one and as an answer