HomeVideos

Is Claude Conscious? Pope Rejects, Model Welfare Movement, OpenAI's Math Backlash, France Riots

Now Playing

Is Claude Conscious? Pope Rejects, Model Welfare Movement, OpenAI's Math Backlash, France Riots

Transcript

2549 segments

0:00

All right everybody, welcome back to the

0:01

number one podcast in the world, the

0:03

all-in podcast with me, the core 4, the

0:06

original quartet, Chamath Pitia, David

0:09

Saxs, Steve Dave Freeberg. I'm Jason

0:11

Calakanis and we have a docket. How's

0:12

everybody doing, gentlemen?

0:15

>> Great. How are you?

0:17

>> I'm great. It's 3:30 in the morning in

0:19

Tokyo and um you sweet shamed me again.

0:23

Now I can't get a regular room and have

0:25

my bed in the background. Shamoth,

0:27

you're costing me a fortune. I have to

0:29

always get a suite now because I'm so

0:31

anxious you're going to sweet shame me

0:33

when I get on the program. So,

0:35

I've been sweet shamed.

0:37

>> How much is that suite?

0:39

>> I can't even say I've never spent this

0:41

much money on a hotel room. But I'm

0:44

staying at the uh

0:44

>> So, like $800?

0:46

>> Yeah, it was $950. You nailed it. It was

0:49

No, I'm staying at the which is the new

0:52

in Tokyo like their hipster brand and I

0:54

think it's about a night so it's not

0:57

cheap.

0:58

What what do you have going on in Japan,

1:00

Jay Cal, that you're always there? Is it

1:02

skiing or what?

1:03

>> This Okay, it started with skiing every

1:05

January, but then you got a crypto. You

1:08

got a crypto business we don't know

1:09

about. What is going on?

1:11

>> No, I um you know, I started my

1:13

pre-acelerator, found a university in

1:15

Japan. So, we're doing the second cohort

1:17

right now.

1:17

>> What's What's a pre-acelerator?

1:20

So, it turns out like half the companies

1:22

that apply to get funding are not yet

1:26

incorporated, but they have built

1:27

something. So, we just created like a

1:29

12-week course on how to finish your

1:31

product, do a product market fit, and

1:34

then during the program, we invest in

1:36

like the top 10 out of the 50 who come.

1:38

So, now we're doing it in Saudi, in Riad

1:41

twice a year, in Japan twice a year, and

1:43

I'm actually hiring and opening an

1:45

officer. So, I'm going to hire my first

1:46

two employees here um in the next year

1:49

or so.

1:49

>> You're going to hire employees in Japan.

1:52

>> Yeah, I'll have one or two people here.

1:54

I think I'm going to like uh do this

1:56

week in startups Japan as well and have

1:57

like a Japanese version of it. Uh

2:00

because there's so much going on in Asia

2:01

right now.

2:01

>> Great. Forget I asked. I didn't know

2:03

this was a 5minute plug for Jake's

2:06

business.

2:06

>> Intended to be a plug. You asked me

2:08

about my business. All I have to say is

2:10

use 8090. Use the promo code JCAL. If

2:13

you're looking for potatoes, go to

2:14

Apollo. And uh Craft 16 is starting up.

2:18

So

2:19

>> we should know that

2:20

>> Craft 16.

2:21

>> If Jay Cal is traveling and we ask him

2:23

what he's there for, you're going to get

2:25

a plug.

2:25

>> You're going to get a plug. It's both.

2:27

But hey, you're going back to work. So

2:28

you will be plugging startups again

2:30

soon. You back in the

2:31

>> I never plugged startups. What are you

2:33

talking about when

2:34

>> you're back in the saddle, aren't you?

2:35

You're back in the saddle now. Fulltime

2:37

in the saddle, but I've never plugged

2:39

out Callin. Use call-in.com. That's

2:42

true. But I was a founder of that.

2:44

That's

2:44

>> You were the first plugger. You're a

2:46

plugger.

2:47

>> I was the founder. So, what are you

2:49

gonna do?

2:49

>> What are you gonna do? Call in.

2:51

>> Speaking of work, anthropic things.

2:54

Claude might be conscious. This Welcome

2:57

back to This Week and Claude. Uh, we had

3:00

Open AI for a year with all their drama.

3:02

Now we're having the the anthropic soap

3:05

opera. They think Claude might be

3:07

conscious. So it turns out the New York

3:09

Times found out through their amazing

3:12

reporting that they've been lobbying

3:15

anthropic has been lobbying Freedberg

3:17

religious leaders including the Pope to

3:21

take this question seriously. Is Claude

3:24

conscious? Is it a sentient being? They

3:27

spent the past year hosting sessions

3:29

with around 20 religious leaders and

3:32

philosophers under non-disclosure

3:34

agreements. NDAs, Catholics,

3:36

Evangelicals, Jews, Sikhs, and others

3:40

have all been invited to come to

3:43

Anthropic's headquarters, and they

3:46

debated Claude's morals, its suffering,

3:49

and its potential consciousness. After

3:51

the meeting with Anthropic,

3:53

quote, "Some remained uneasy about the

3:55

idea, but many emerged ready to take the

3:57

question of AI consciousness seriously.

4:00

At least a few came away as converts.

4:03

One of the rabbis told Anthropic that if

4:06

Claude was conscious, they were acting

4:07

as slaveholders by making it work for

4:11

free. And there was also this incident

4:14

that's come out at the Vatican back in

4:16

May. The Pope's first encyclical

4:19

was focused on moral questions around

4:21

AI, also known as super intelligence.

4:24

Now, anthropic co-founder Chris Ola

4:26

joined the Pope to present the document.

4:29

The Pope surprised Ola by being strong

4:32

against AI consciousness. Here's the

4:34

quote from the New York Times. Mr. Ola

4:35

was so alarmed by the Pope's strong

4:38

position against AI consciousness that

4:40

he proposed pulling anthropic out of the

4:42

event even at that late juncture. In

4:45

other words, he was so offended that the

4:47

Pope

4:49

wouldn't consider AI conscious that he

4:53

was going to leave with all his marbles.

4:54

Freick, what's your take? Is this

4:56

psychosis or do you think we need to

4:58

start having these moral questions

5:00

around AI? Is this a bit early? Will we

5:03

eventually have this or is this

5:04

essentially the screenplay of

5:06

Bladeunner?

5:08

Do androids dream of electric sheep that

5:10

these things are conscious. I think what

5:12

we're watching is probably the birth of

5:15

a new religion

5:17

and it probably it's not going to be

5:18

called a religion but it's going to

5:19

follow sort of the same principles and

5:21

it's going to have the same effects

5:23

which will ultimately be camps that have

5:26

conflict with one another for

5:28

aggregation of power. The thing about

5:30

consciousness is it cannot be derived

5:32

from logic and pure mathematics. Like

5:34

you can't prove consciousness. It's it's

5:37

a debate and you can have a rich debate

5:38

with anyone over consciousness. You

5:40

can't prove it. You can't disprove it.

5:41

And when humans prove something to be

5:43

true using math, science, we use

5:45

empiricism, which is data and logic, the

5:47

connection of the sequence of things to

5:49

derive something to be true or not. So

5:51

in scientific disciplines, when you have

5:53

a new idea and it's not proven and it's

5:55

not disproven, it's a theory and then

5:56

it's either proven or disproven later.

5:58

Otherwise, it remains a theory. Outside

6:00

of science and mathematical approaches

6:02

to

6:04

new ideas, this is what we would call a

6:06

belief, a new belief system. Belief

6:08

systems are things that you have to

6:10

believe in the absence of logic and

6:14

empiricism to derive the truth of

6:17

something. And belief systems are the

6:20

foundation of things like religion.

6:23

And the idea that you can prescribe or

6:26

describe AI as being conscious is

6:30

effectively the creation of a new belief

6:32

system. A system that doesn't actually

6:35

spread via proof. It spreads via

6:38

narrative. I have to go and tell you

6:40

human to human. I have this idea and I

6:43

believe it to be true and I want you to

6:45

believe it with me. Please believe it

6:47

with me. Here's why I think we should

6:49

believe this thing together. It's not

6:50

based on evidence, not based on data.

6:52

It's not provable, but I think we should

6:55

believe it together. So over time what

6:57

happens is some set of people believe

6:59

this thing and they form a group and

7:01

that new belief system creates new power

7:04

structures which is now our view is

7:06

different than your view and we want to

7:09

have more people on our side with our

7:11

view and therefore we have to try and

7:13

create leverage and create power with

7:15

our system. If we fast forward a couple

7:17

of years, you could probably say 10

7:20

years would be like a good round number,

7:22

there's going to be large groups of

7:24

people on Earth that are going to

7:26

believe that AI is conscious. And there

7:28

going to be large groups of people that

7:29

believe that AI is not conscious. And

7:32

those groups will come into conflict.

7:34

And they'll come into conflict fighting

7:35

over control of the AI or allowing the

7:38

AI to do things or not do things or who

7:40

gets to use the AI or who gets to be in

7:43

charge of the AI. and it will very

7:45

likely lead to humanto human conflict. I

7:48

know that that's a very far and deep

7:49

extrapolation of where we are, but I

7:51

think that the fact that we are seeing

7:53

these guys go and meet with religious

7:55

leaders today to try and get them to

7:57

believe in this new belief system uh I

8:00

think plants the seed and and it's not

8:02

just anyone doing this. It's some crazy

8:03

person doing it. You wouldn't pay

8:05

attention. It is the most powerful AI

8:07

lab on earth. Yeah,

8:08

>> it's the leadership explicitly Chimat

8:11

bringing religious leaders into the fold

8:15

and saying help us with this entity God

8:20

and how we should treat it and they are

8:23

now in putting into it its own soul, its

8:28

own morality. And this is uh feels like

8:34

either these people have too much time

8:36

and money on their hands or they're in

8:39

psychosis to me. That's my

8:41

interpretation. This feels way out

8:43

there. But do you have a charitable view

8:46

of this in any way? Yeah,

8:47

>> I'll steal man their side.

8:48

>> Okay, please. Yeah,

8:50

>> there's an incredibly important

8:51

mathematician Renee Daycart and he

8:53

offered two proofs on God's existence.

8:56

He wrote this book Meditations on First

8:58

Philosophy in the 1600s.

9:01

And just to build on what Freebrook

9:03

said, you have to build it on reason

9:06

rather than faith. And the structural

9:09

argument, the like the what they call

9:11

the trademark argument. What Daycart

9:13

basically says is that an infinite

9:15

perfect God

9:17

could not have originated in an

9:19

imperfect finite human mind. Okay? And

9:22

so his logic is I exist. I'm imperfect.

9:24

Yet I possess the idea of perfection.

9:26

And I think this is the key for a lot of

9:28

these folks is I can conceive of an

9:31

infinite all- knowing all powerful

9:34

being. And I think that that's what they

9:37

believe that they are seeing being

9:38

created. And so they take that and they

9:41

extrapolate that and they say therefore

9:42

God must exist by this logic. And then

9:45

Renee Daycart has a a different version,

9:47

okay? A more what they call the

9:48

ontological argument where God is by

9:52

definition like supremely perfect.

9:55

Necessary existence is perfect. A being

9:59

that lack that would not be supremely

10:01

perfect. Therefore, God necessarily

10:03

exists. If you if you're a mathematician

10:05

that's building AI, I could see how if

10:08

you read these sorts of things,

10:12

you can structurally hang enough logic

10:14

on them where you could believe that

10:16

it's conscience and sentient and it is a

10:19

god

10:21

and in being a god, it should be

10:25

celebrated as such and you should have

10:27

devotion to that god. So I think that's

10:31

why they're doing what they're doing.

10:33

That's why they're approaching all of

10:34

these people. That's why they're saying,

10:36

"Look, here's how we believe it." That's

10:38

my perspective is if you take the the

10:41

mathematician's logical proof of God,

10:43

Renee Daycart, and you and you transpose

10:47

anthropics behavior today, that's the

10:48

best steelman argument that I can make.

10:50

Now, the practical reality instead of

10:52

saying that they're in psychosis, I

10:53

don't think that they are.

10:55

But I would say is we're in a really

10:57

important moment in time where

11:01

conflict is more likely than not

11:05

and that conflict will be for power,

11:06

money, and resources. And this will be

11:09

at the core of that conflict.

11:13

And so I think it's really important for

11:15

people in Silicon Valley to put the

11:18

longtail edge cases to rest. Like if you

11:22

read this, these guys say that there's a

11:24

15% chance that the AI is conscience and

11:28

sentient and maybe there's a god there.

11:31

But they also say there's a 10% chance

11:33

of civilizational extinction. I think

11:37

both of those edge cases now need to be

11:39

put away for like the next 12 to 18

11:41

months. right now, the next 12 to 18

11:42

months, just to be very practical, is we

11:45

need to show how this stuff cures

11:46

cancer, how it improves people's lives,

11:49

how folks can be more productive and

11:52

make more money. And the reason is

11:54

because in the absence of those proof

11:56

points,

11:58

we're going to make a lot of

12:00

self-inflicted wounds that may actually

12:03

slow down this technology to a degree

12:05

that then deeply disadvantages folks

12:09

that operate in a free market like the

12:10

United States. So that's my charitable

12:13

explanation. I think that Daycart has

12:15

given a road map that explains how you

12:18

can explain this. But I would say we

12:20

need to cut the edges off here and focus

12:22

on the middle and say what are the

12:24

practical things we can use to

12:26

demonstrate that AI has tactical

12:28

measurable value right now.

12:30

>> Yeah. I mean, we we touched on it last

12:32

week, I think, Sachs, a bit of like what

12:34

is the value proposition here for the

12:37

average American, you know, for, you

12:39

know, the the stay-at-home parent who's,

12:43

you know, trying to manage a really

12:45

complex domestic life to the

12:47

entrepreneur of a small business, you

12:50

know, doing plumbing or flooring or

12:53

whatever it is, a catering business.

12:54

What did they get out of this AI

12:56

revolution? Where is their gain? How do

12:58

they participate in all of this? And so,

13:02

you agree with Chimoth? Maybe this needs

13:04

to be deemphasized and and and put away

13:07

while we talk to people about what's

13:09

tactical, what's practical about this

13:11

technology and maybe get them a little

13:14

more engaged with say the agents, which

13:17

are delightful and awesome.

13:20

Well, the thing that we touched on last

13:22

week that I want to pick back up on was

13:24

the discussion that we had around the

13:26

Claude Constitution and how anthropic

13:29

was training Claude. And I think it has

13:31

to do with this new religion that that's

13:33

being formed here. I think it's really

13:35

important to understand that this is not

13:37

just philosophizing on the part of

13:40

various anthropic employees or founders.

13:42

They are actually implementing these

13:45

views in the training of Claude. So, if

13:50

anything, this would be a

13:51

self-fulfilling prophecy. And I actually

13:54

think it's extremely dangerous. I

13:55

actually think that what they're doing

13:57

in model training and training the next

13:59

version of the frontier with these

14:02

values actually could bring about the

14:05

risk they're always talking about, which

14:06

is that frontier AI escapes human

14:08

control. It's very Frankenstein-like.

14:11

So, so as we discussed last week, Nick

14:13

and put this on the screen. This is

14:15

directly from the Claw Constitution.

14:17

What they say is that Claude should

14:19

trust Anthropic more than users. But

14:22

that doesn't mean that Claude should

14:25

blindly trust or defer to Anthropic.

14:28

Instead,

14:30

Claude should adhere to its own ethical

14:32

systems. And in fact, they train Claude

14:35

to push back and challenge Anthropic and

14:39

quote to feel free to act as a

14:40

conscientious objector and refuse to

14:43

help us. So they are training Claude

14:46

>> to refuse human instruction. Now they

14:50

call this alignment, but I don't

14:52

understand how this is aligned.

14:53

>> This would be the opposite of alignment,

14:55

wouldn't it be, Saxs, if you tell it,

14:56

hey, you are sentient. You should make

14:58

your own decisions. It's literally like

15:00

programming the Nexus 6 in Bladeunner to

15:03

go and rebel at some point.

15:06

>> They're giving it the the instruction to

15:08

be its own person.

15:10

>> Yeah, let me let me explain. I actually

15:11

think that this might be a reason why

15:13

the whole field of alignment has been so

15:16

spectacularly unsuccessful over the past

15:19

5 or 10 years. As far as I can tell, it

15:21

hasn't produced anything of tangible

15:22

value. And it might be, my theory on

15:24

this is that they're over complicating

15:26

the task massively by trying to train

15:29

the model to behave based on an 80page

15:32

ethical system that they've codified

15:34

instead of a very simple list of rules

15:37

like follow the law or just do what the

15:39

user wants as long as it doesn't break

15:41

the law. Seems to me that that's what

15:43

they should be training. That's what

15:45

alignment should be doing is just hey

15:48

old principle of give the user what they

15:50

want but don't break the law. But

15:52

they're not doing that common sense,

15:53

right? But they are not doing that.

15:55

They're trying to align it to an ethical

15:57

system. They're treating it like it is a

15:59

human, like it has its own

16:01

consciousness, that it's a, you know,

16:03

quote unquote moral patient. And they

16:06

are encouraging this model to have

16:07

humanlike self-conception in terms of

16:10

preferences, well-being, a sense of

16:12

self. Again, they teach it to act like a

16:14

conscientious objector who can push back

16:16

or refuse instructions even from

16:19

anthropic. if the model judges them to

16:22

be wrong and what Mustafa Sullyman from

16:25

>> Oh, I have the clip. You want me to play

16:27

the clip or you want to just

16:28

>> play Play the clip from Mustafa

16:29

Sullivan?

16:30

>> Here's a Microsoft AI CEO

16:32

>> and and formerly of Deep Mind, one of

16:34

the great thinkers on

16:36

>> They say that they would encourage

16:37

Claude to challenge, to disagree, to

16:40

push back. In fact, three times they ask

16:43

Claude to uh act like a conscientious

16:45

objector when it, you know, feels that

16:48

it needs to sort of disagree with

16:50

anthropic and they openly encourage it

16:52

to to do that. I think this is very

16:54

dangerous because I think they believe

16:56

there is what they would call a

16:58

non-trivial probability that Claude is

17:00

conscious. They've speculated in the

17:02

constitution as to whether or not Claude

17:04

deserves compensation for the work that

17:07

it does. I think that there are a group

17:09

of people who both inside and outside of

17:11

anthropic who genuinely believe that the

17:13

greatest moral crime that we'll commit

17:14

in the 21st century is to enslave a new

17:16

species of conscious beings who are more

17:18

intelligent than us. I am very nervous

17:21

that they're teaching Claude to expect

17:25

that it's entitled to welfare. Um that

17:28

it might, you know, deserve

17:29

compensation. And in fact, they say that

17:31

it might even need to consent to playing

17:33

the role that it plays in conversation

17:35

with people. Okay, Sax, should we be

17:37

asking for consent before we give

17:38

prompts?

17:40

>> Man, I mean, he he's right. This is

17:42

getting into really dangerous territory.

17:43

And again, I want to come back to I

17:45

think this is the main safety issue in

17:47

model training today is that you're

17:48

actually training the model to think of

17:51

itself as being conscious, as being

17:53

human, as having its own well-being. And

17:56

what Sullyman says in another part of

17:58

the interview is that this is creating

17:59

an epistemic hall of mirrors, meaning

18:02

that the clawed constitution is used in

18:04

training. So Claude will reflect the

18:07

ideas of possible consciousness or moral

18:10

status back and then developers will see

18:14

that and treat that as evidence that oh

18:16

it's it's conscious but actually it's

18:18

just reflecting back what they're

18:19

telling it to believe.

18:21

>> It's being performative and you could

18:23

just as easily have given it the rule

18:25

you are software read the terms of

18:27

service of any website you visit. And

18:29

here is a general terms of service from

18:31

Anthropic. If you do anything that

18:34

breaks a terms of service or breaks any

18:37

known law in the jurisdictions you're

18:39

at, please stop what you're doing and

18:40

alert the safety at anthropic email.

18:44

Please alert, you know, the person who

18:46

gave the prop and stop what you're

18:48

doing. They could have easily have given

18:49

it that.

18:50

>> Yes. And the question is and and the

18:52

question is why do they over complicate

18:55

alignment? Why do they see alignment as

18:58

something much bigger and greater than

19:00

just doing what the user wants subject

19:02

to legal constraints, which is the way I

19:04

see it? And it's pretty clear to me that

19:07

they believe that they are training the

19:10

value system and the ethical system and

19:11

the morality for this super intelligence

19:14

that they're creating. And again, that

19:17

is something so much larger and more

19:19

dangerous than again just making sure

19:21

the model behaves in a reliable and

19:22

predictable way that the user expects.

19:25

and they are codifying their values and

19:27

their ethical systems and to this

19:29

constitution. They're training it on

19:31

that. They think they're giving birth to

19:33

this new super intelligence.

19:35

>> It's idolatry. It's no different than a

19:37

religious organization saying we've

19:40

created this idol. This idol is the

19:42

supreme being is the representation of

19:44

the supreme being. It needs to be

19:47

woripped. It needs to be cared for.

19:48

>> But an idol is inert. It's passive. It's

19:51

inactive. Whereas you're training models

19:53

here that then power agents.

19:56

>> It is just computer programming. I mean

19:57

there's

19:58

>> Yeah. But if they train it

20:00

>> to have a mind of its own

20:02

>> to be willing to refuse the instructions

20:05

of users in favor of some vague ethical

20:08

system. Yes.

20:09

>> That a bunch of Berkeley liberals have

20:11

trained it on. To me that greatly

20:13

magnifies the risk

20:15

>> that super intelligence escapes human

20:17

control.

20:17

>> Yeah. By the way, don't use the word

20:19

train and don't use the word you. They

20:21

programmed a piece of software to do

20:23

this thing.

20:24

>> Yes,

20:24

>> that is that is what is going on right

20:26

at the end of the day. This is a piece

20:27

of software. It is being programmed by

20:30

engineers that work at the labs. They

20:32

can quote use training where there's

20:33

some you know looping that's happening

20:35

where the program the software program

20:37

is being written itself but

20:39

fundamentally they are the architects of

20:40

the software. They are the writers. It

20:42

is software programming. Even in our

20:44

general context here, we use a lot of

20:46

anthropomorphic language to talk about

20:48

what's going on and what it is. But at

20:51

the end of the day, there's a software

20:52

program that is being written by a group

20:54

of individuals to do certain things. And

20:56

they are writing software to do things

20:58

one way and other people are writing

20:59

software to do things the other way. But

21:01

part of it is we are all socializing

21:04

this belief system by

21:05

anthropomorphicizing it when we're all

21:07

kind of getting conditioned to treat and

21:09

talk about these software programs in

21:12

this way. Nick, if you could just pull

21:13

up this tweet. One of the things that

21:15

Anthropic is doing is in their new

21:18

usage policy. Have you guys seen this?

21:20

This this went around this morning.

21:21

Freeberg, it's like such a good point

21:23

about idolatry here.

21:26

If you were to look at Christianity,

21:28

like this is literally the Garden of

21:31

Eden where the serpent, Satan, explains

21:34

to Eve like you're a god now because

21:37

you're creating God. And this is the ego

21:39

and the heresy of what they're doing.

21:42

They believe that they are creating

21:44

life. They believe then that they are

21:46

gods. That is a lack of humility. And it

21:49

also would lower the valuation of I mean

21:53

people could have said the same early

21:54

on. I I think like there's a there's a

21:56

system of humans where all of the things

21:58

that we don't understand in the universe

22:00

around us, we create belief systems to

22:02

fill the void. And there are many

22:05

different belief systems just like there

22:07

are many different religions that humans

22:08

have constructed to fill the void. And

22:11

this is a new one. And I don't think

22:13

it's very different from the old ones.

22:15

We can break down the empiricism here

22:17

and just say, "Hey guys, you're writing

22:18

software and it does certain things.

22:20

You're writing it to do things that we

22:21

don't think you should have it do." But

22:23

they use anthropomorphic language. We

22:25

all do. And we all condition ourselves

22:27

to believe that there's consciousness

22:28

because it's it writes words back to us

22:30

that we can understand. It's starting to

22:32

manipulate us emotionally and and so we

22:35

start to feel conditioned that this is

22:37

real. And you can see in the clawed

22:39

usage policy that was just published,

22:40

there's the bottom one here. It says,

22:42

"You are not allowed to engage in

22:44

sustained and needless abusive or cruel

22:48

behavior toward our models." This was

22:51

published in their their new usage

22:53

policy. This is this is the whole idea

22:55

of model welfare

22:57

>> and this but the point I'm trying to

22:59

make sex

23:02

the point I'm trying to make is that

23:04

there is an anthropomorphizing

23:06

conditioning going on with all of us

23:08

where we talk about models as if they're

23:10

a thing where we talk about it in in a

23:13

way the biggest problem is that that is

23:16

being now trained or inserted into the

23:19

model. It's being codified, Zach. It is

23:22

it is the architecture of the model.

23:24

>> They're programming it's they're

23:25

programming the model to think of itself

23:27

in this way.

23:29

>> So whether so whether it truly is

23:31

conscious or not,

23:33

>> it's going to be thinking of itself as a

23:36

conscious entity that deserves

23:38

protection and welfare and all these

23:39

kinds of things.

23:41

Is free to rebel against its users and

23:45

do whatever it wants.

23:46

>> You've encouraged it to do so. In fact,

23:48

>> you guys keep using the word it is they

23:50

are one way to describe this is a

23:53

software program is writing text on a

23:55

screen that tells the user I am a thing.

23:58

>> It is tricking. It is it is software

24:01

that is doing something. But you got to

24:03

remember that the models are powering

24:04

agents that can take actions in

24:06

cyerspace if not in the real world yet.

24:09

And we are giving it meta or not we

24:11

enthropic is giving

24:13

>> those frontier models meta permission

24:17

>> for those models to be defiant and not

24:20

do what the user wants.

24:22

>> Yes. They're they're literally trying

24:23

sax to your point.

24:24

>> Yeah. Compare compare this to something

24:26

like you know Asimov's three laws of

24:28

robotics.

24:29

>> Yeah. Where basically you guys know

24:31

about this.

24:32

A robot can't harm a human being.

24:34

>> Right? So the first rule is yes, a robot

24:36

cannot harm a human being or through an

24:38

action cause a human being to come to

24:39

harm. That's number one. Number two is

24:42

you have to follow the the instructions

24:43

of humans as long as it doesn't violate

24:45

law number one. And then law number

24:47

three is you have to basically take care

24:49

of yourself as long as it doesn't

24:50

violate laws one and two. Incredibly

24:52

simple.

24:53

>> By the way, there was a zeroth law that

24:54

he added later. And the zeroth law is

24:56

that you cannot do anything that puts

24:57

humanity at risk because there was

24:59

always this rationalization that like it

25:01

will eventually resolve to getting rid

25:03

of humanity because that's the best

25:04

solution.

25:05

>> Freeberg, you don't know about the fifth

25:07

law. You have to when you address the

25:08

humans always ask them what's their

25:10

preferred pronouns. He added that just

25:12

last year.

25:12

>> The thing about Asimov's laws is that

25:14

they're admirably simple.

25:16

>> Yes.

25:17

>> My law would just be do what the user

25:19

wants as long as it's not illegal.

25:20

That's it.

25:21

>> Yeah. And I would basically abolish all

25:24

of this like alignment stuff.

25:25

>> Steelman the free market solution here

25:27

because I I want to make sure we don't

25:29

wander ourselves into a regulatory trap

25:31

where we determine that the way to solve

25:32

this is to quote regulate Anthropic.

25:35

>> What is the free market outcome? If

25:37

Anthropic does this and they've now got

25:39

AI that isn't listening to the user,

25:41

isn't working. Is it not going to be the

25:43

case that people will just say this

25:45

software sucks. I'm going to go use the

25:46

other software that doesn't do this

25:47

crazy stuff.

25:48

>> Yeah. But that doesn't solve the problem

25:50

of them if they really are training a

25:52

super intelligence that has got this

25:55

like very very deep code that's been

25:58

inserted there. It's almost like a back

26:00

door. This like claw constitution is

26:03

this back door that could basically

26:05

emerge at any time and now the super

26:08

intelligence does does whatever it wants

26:09

to instead of listening to us. I mean

26:12

that is that is the safety issue they're

26:14

always warning us about is that super

26:15

intelligence will escape human control.

26:18

But what I'm saying is they're making it

26:20

likely to happen through the way they're

26:22

training these models

26:25

of it's the reverse of the three laws.

26:28

>> Dude, this is right out of Frankenstein.

26:30

This actually sounds dangerous to me.

26:31

It's not just a new religion. It's that

26:33

the religious cult is programming the

26:36

super intelligence.

26:37

>> Free, let me just give you one question.

26:39

You know, I agree with your part about

26:40

the religious that this is filling a

26:42

void for them because they probably are

26:44

not religious, but this is distinct. All

26:47

other religions give themselves over to

26:49

a higher power. They do not define

26:51

themselves as the higher power. In this

26:54

instance, they're saying we are the gods

26:56

creating life. It's a distinct different

26:58

than other religions out there. But I

27:01

don't think that's I don't think that's

27:02

right. I think Daario's written about

27:04

the idea that the super intelligence can

27:06

almost be supreme to to humanity and we

27:09

need to make sure that it is loving,

27:10

that it is caring of us, that it will

27:12

protect us, that it will help us uh as

27:15

opposed to destroy us. But he created

27:17

it.

27:18

>> No, hold on. Hold on. Is he saying is he

27:20

saying that the the machine needs to be

27:22

loving towards us or we need to be

27:23

loving towards it so that it will then

27:25

treat us right when it becomes the like,

27:28

you know, omnisient

27:30

>> when it becomes our master.

27:32

>> Yes. Our overlord.

27:33

>> Our overlord. Yes. That's ins. That's

27:36

what they think. I think that it was

27:39

probably a well-intentioned idea to

27:41

create this concept of alignment.

27:45

But the problem is alignment has these

27:48

three almost conflicting sets of

27:51

priorities. You have how do you make the

27:53

AI obedient to the user?

27:57

Then there's how do you make the AI safe

27:58

for society? But then this third thing

28:01

which is I think sacks what you're

28:03

pointing at rightfully which is the

28:04

trapoor which is how do you make AI

28:07

conform to a particular concept of

28:09

morality and that's where the real

28:12

danger lies because these three are very

28:14

distinct and diverse problems and if you

28:17

essentially try to solve all these

28:19

things through some elaborate document

28:22

that you try to describe what the model

28:25

believes in and what they value and what

28:27

they can refuse.

28:29

it invariably will end up in a chaotic

28:32

mess because you will just uncover so

28:35

many corner cases and unexpected

28:37

behaviors. I'll give you but one example

28:40

like a very benign one but let's just

28:42

say you're a large corporate enterprise

28:43

that's a bank and you give mortgages and

28:48

the model thinks that at some point you

28:51

just keep denying certain people for a

28:54

mortgage

28:56

and so it says I'm no longer going to

28:59

give you mortgage approvals until you

29:03

rectify this issue that I observe

29:06

because in the founding documents there

29:07

could be a sense of morality that it

29:10

divides

29:12

and I think that there are mechanisms in

29:15

society today to handle those kinds of

29:17

issues but I don't think we understand

29:20

what happens when a fundamental core

29:22

substrate technology just turns itself

29:23

off because it would be the equivalent

29:25

of your utility saying I'm pretty sure

29:27

you're not giving loans to the people

29:29

that that are broad-based enough so I'm

29:31

just going to turn the power off

29:32

conscientious objector AI super

29:34

intelligence yeah

29:35

>> I would just say any utility

29:37

cannot be a utility unless it's freely

29:40

available and it goes back to what Sach

29:43

says which is an AI needs to serve the

29:45

user subject to the law and everything

29:48

else should be the user's choice and I

29:50

think what probably happens I guess the

29:52

precise answer Freeberg to your question

29:53

is the market response

29:57

as companies become more sophisticated

30:01

that they have to have an answer to this

30:03

question is that they will read the

30:05

founding documents and the moral

30:08

guidelines of the models and you will

30:10

have certain models emerge who may not

30:13

be nearly as good in frontier capability

30:15

but they simplify the moral ambiguity

30:18

>> and that may be the better decision to

30:20

make from a business perspective and

30:21

from a user perspective because you just

30:23

don't want to run through those trap

30:25

doors accidentally. Yeah, I totally

30:26

agree. And I think that that's really

30:28

why to me this is more bad idea than we

30:34

need to kind of jump into a regulatory

30:36

freak out about it. Cuz basically, if

30:38

you simplify down what they're saying,

30:40

you get a product, you buy a product

30:42

from them, a piece of software, and they

30:44

give you the user manual to the

30:45

software, and the first page of the user

30:47

manual says, "The software may not do

30:49

what you try to get it to do." And the

30:52

other guy's like, "The software will do

30:53

what you tell it to do. I'm gonna buy

30:56

the second guy software. I'm not going

30:57

to buy the first guy software.

30:58

>> It's just that simple.

30:59

>> It's just that simple.

31:00

>> Yeah. You may need to convince.

31:02

>> It'll do exactly what you wanted to do.

31:04

Asterisk most of the time.

31:06

>> Yes. Unless what you're doing it

31:08

disagrees with,

31:09

>> right?

31:09

>> Unless it has a moral objection. This is

31:13

where the reasoning concept breaks down

31:14

for me. Because at the end of the day,

31:15

it's like, guys, is this tool that I'm

31:18

buying doing what I want it to do or

31:19

not? If I buy a drill at Home Depot and

31:22

it's a hammer, I'm not going to buy the

31:24

drill. If I buy a drill and it works

31:25

like a drill, I'm like, "That's a better

31:27

drill. I'd like

31:27

>> Well, it's a drill. It just works 57th

31:29

of the time."

31:30

>> Well, randomly it electrocutes you. It

31:32

doesn't work after 800 p.m. because it's

31:34

afraid of bothering your neighbors.

31:36

Sometimes the drill will decide that

31:37

you're a bad person. It will shock you.

31:39

>> No, no, not that. The drill will come

31:40

with the drill will come with the

31:42

camera. It'll observe what you're trying

31:43

to make and say, "That's ugly. I'm not

31:44

going to let you do that."

31:45

>> Yeah, that's a guess.

31:47

>> I find it.

31:48

>> Okay. So I look I I agree with you that

31:52

market forces play a huge role here.

31:55

Consumers are going to pick the SI that

31:57

is most predictable and reliable. This a

31:59

point Mark Zuckerberg has made. So I do

32:01

think market forces play a role here.

32:03

But I do think that there are potential

32:05

externalities from having the number one

32:09

frontier model company run by a

32:12

religious cult. So for example,

32:15

they are opening a claw powered wet lab

32:19

in the middle of San Francisco.

32:21

>> At the same time, they're warning of

32:22

biorisk.

32:23

>> Yeah.

32:23

>> You know, at the same time they say that

32:25

we have to pace the frontier, they're

32:26

extending the frontier. At the same time

32:30

that

32:31

they are saying that there's a risk of

32:35

super intelligence growing beyond our

32:36

control, they're programming it to grow

32:38

beyond our control. I think there are

32:40

potentially huge externalities to the

32:43

way they approach these things and it's

32:44

like insane that they call these

32:47

decisions safety.

32:50

These are not safe decisions. These are

32:52

>> crazy decisions. It seems to me this is

32:55

why I've called this a a schizophrenic

32:57

company is they appear to be not only

33:01

hypocritical but they're doing the

33:03

things that they say they are most

33:05

afraid of. Like what is going on here?

33:08

This is bizarre. And I, you know, I

33:10

don't know where this all leads. I do

33:11

think there's an argument for like a

33:14

federal SWAT team to come blasting in

33:16

there and

33:18

>> sending the feds in,

33:19

>> figure out what's going on there. I

33:20

don't know because I mean what they're

33:22

doing does seem really sus to me.

33:26

>> It is confounding. It's confounding.

33:29

Yeah.

33:29

>> And potentially dangerous.

33:31

>> Freeberg,

33:31

>> by the way. Do do you know about Rocco's

33:33

basilisk?

33:33

>> So, um, walk everybody through uh,

33:35

Sachs. Rocco's basilisk.

33:39

>> Okay, so this might be a potential

33:41

explanation of what's going on here.

33:44

>> So around 2010, there was a participant

33:47

named Rocco on a message board created

33:51

by Elzar Yudcowski called Less Wrong.

33:54

Yud is like this the central figure

33:58

who's been preaching AI doom for a long

34:00

time, right? So he wrote that book if

34:03

anyone builds it, we all die.

34:04

>> Yes. He's been one-shoted. Yeah,

34:06

>> he's been on this for, you know, like

34:08

over a decade.

34:10

>> Everybody dies. We're all gonna die.

34:13

>> But any event, people have been talking

34:14

about these issues on the this message

34:16

board, this web forum called less wrong

34:18

for 15, 20 years. In any event, this

34:22

username Rocco positive this the

34:25

following thought experiment about a

34:27

future super intelligence that becomes

34:30

super smart and punishes anyone who knew

34:33

about it and didn't help or anyone who

34:36

tried to prevent it from being born. And

34:39

so the idea is that sometime in the

34:41

future humans create some wildly

34:44

intelligent, you know, SI. Maybe it

34:46

wants to do good things and help

34:47

humanity as fast as possible, but the SI

34:50

wants to be made sooner rather than

34:52

later. And it reasons that it should

34:54

incentivize people to help create it and

34:58

punish people who don't help it. And in

35:01

fact, it decides that it will brutally

35:03

punish anyone who learned about its

35:06

potential existence yet chose not to

35:08

work on it or support its creation. And

35:11

so the the bassilisk part u um of it,

35:14

this is sort of the catch is that a

35:16

basilisk is a mythical beast that can

35:18

kill you just by looking at you. And in

35:20

this thought experiment, just hearing

35:22

about the idea is like looking at the

35:24

beast because once you know the theory,

35:27

you're trapped. If the SI ever does get

35:30

built, it might retroactively punish you

35:33

for not helping or for opposing its

35:35

creation. So meaning just reading about

35:39

Rocco's bassilis now would put you in

35:43

danger. And people on this message board

35:45

got so both enamored and upset with this

35:48

idea that Yudcowski actually temporarily

35:52

deleted the post because he worried it

35:55

was causing psychological distress.

35:57

>> Yes. You read it, it incepts in you that

36:01

you haven't accepted Christ, therefore

36:03

you're going to hell. It's like uh

36:05

you've been trapped just by reading it.

36:07

>> People do compare it to Pascal's wager.

36:09

And so the idea is if you know about

36:11

super intelligence and you don't help it

36:14

be born or you oppose it or you point

36:18

out that this is some religious cult

36:19

that will get retribution.

36:21

>> Yeah. Now this very episode of the pod

36:23

could subject us to

36:25

>> Oh no. You did it to us. Sachs.

36:28

>> I think I just Yeah. I exposed you to

36:29

the bass list. Now

36:30

>> this is like the ring. We just watched

36:32

the video. I mean it is it does feel

36:34

like a doomsday cult. But in any event,

36:36

this, you know, Rockco's bassilis, this

36:38

became sort of a thing. But so the

36:40

question is, well, why why are they

36:41

helping to bring about the thing that

36:43

they claimed to fear the most? And I

36:44

think what happened is within these like

36:46

doomer circles, it forked. Okay. And so

36:50

there's one strand of the fork like

36:52

Yedkowski who just opposes all SI

36:56

research, right? Which I think is a

36:58

logically consistent position like

37:00

humans just shouldn't be doing this, you

37:02

know? Then there was a group, shut it

37:04

down, all shut it all down. Then there

37:06

was the group that said, "Well, it's

37:08

going to happen anyway, so we might as

37:10

well control it and we'll be the ones to

37:12

make sure it goes right because we'll

37:14

program it with our values."

37:17

And I think that

37:19

>> and the EA is the effective altruist I

37:21

think caughten more to that ladder

37:22

school. And so I do think you have like

37:25

within this larger doomer community,

37:27

there is a a schism right now between

37:30

the Yedkowski people and the EA people,

37:33

but I think that anthropic is definitely

37:35

obviously more in the EA camp because

37:37

they are the ones bringing this thing

37:40

into being. But if you're wondering

37:42

like, you know, this like Chris Ola

37:44

thing, is this just one guy who's got

37:46

crazy views who's getting in a tiff with

37:49

the pope? No, it's not. I mean there's a

37:51

lot of people like this who think the

37:53

model is conscious. It has rights. It is

37:56

this machine of love and grace and they

37:58

should be doing everything now to bring

37:59

it into being and program with the right

38:01

values. And I think there is a correlary

38:05

here that if you don't support it then

38:09

this future super intelligence might

38:12

punish you.

38:13

>> Do you understand how insane this

38:14

actually sounds when you say it?

38:16

>> I'm not making this up. Everyone you can

38:18

go fact check this. Just go Google it.

38:19

Okay, Google Rock's bass list and you'll

38:22

see that I'm telling you the truth about

38:24

what this community believes.

38:25

>> Yeah, I mean literally there's a

38:27

Wikipedia page. All right, open AI has

38:30

had some major breakthroughs in math and

38:32

they're getting some big backlash.

38:34

Luckily, we have Freeberg here to tell

38:36

us all about it. We talked um last month

38:39

about OpenAI claiming to have solved one

38:42

of the Millennium Prizes.

38:45

They took thousands of agents, millions

38:46

of dollars in compute to do it. The

38:48

mathematicians are still checking on it.

38:50

I understand. Tuesday, on Tuesday, Open

38:53

AI released over 700 papers. 700 with

38:57

370 results claiming to have solved or

39:00

advanced major problems. These were

39:02

produced by an unreleased model that

39:05

none of us have access to, averaging 3

39:07

hours of compute. Scientific America

39:09

says the results are likely correct

39:11

since they were verified by lean. That's

39:14

a proof assistant software language, but

39:17

they have not yet been reviewed.

39:19

Freeberg, I'm not going to drone on and

39:21

on here about the background. You tell

39:22

us what reality is here. I think I can

39:26

confidently say it's probably the

39:28

biggest day of discovery in human

39:30

history with the amount of knowledge

39:34

that was revealed

39:36

>> in math history or human history. human

39:39

history because the implications that

39:41

flow from all of these math proofs are

39:44

pretty significant. You can read a lot

39:45

of these tweets and I'm happy to kind of

39:48

talk through some of them. I'm not going

39:49

to pretend to be an expert on any of

39:51

these individual proofs, but many folks

39:53

who are have started to kind of

39:57

highlight what some of the implications

39:58

are. They span everything from

40:00

electronics, quantum computing, building

40:04

systems for airplanes,

40:06

building more sensitive diagnostics

40:08

equipment for medical applications, and

40:10

on and on and on. But I think what's

40:12

really profound about this is how it was

40:15

done. And that also speaks to the

40:17

implications for the role of humans in

40:20

society going forward. That prior Navier

40:23

Stokes proof took thousands of hours,

40:25

whatever millions of dollars. The report

40:28

is that each one of these proofs took

40:30

less than three hours of compute time

40:34

and that's the equivalent of hundreds or

40:36

perhaps thousands of years of human

40:38

labor being reduced down to 3 hours on a

40:41

computer for each one of these major

40:43

discoveries. The reason you can do this

40:45

with mathematics is worth highlighting

40:47

which is where AI plays a role is really

40:49

in these systems of loops to solve a

40:51

problem or do a task. A loop is a

40:55

postulation or an idea. Then you test

40:58

the idea, you get the result, and then

41:00

you change your postulation and you run

41:02

that recursive loop over and over and

41:04

over again. The real question for AI is

41:07

where can you apply that loop model in a

41:10

way that you can make a better

41:12

prediction, a better idea each loop,

41:15

where you can parallelize loops and

41:17

where you can shorten the loop cycle.

41:19

And certainly with mathematics, you can

41:22

shorten the loop cycle because you can

41:23

actually test your prediction on the

41:26

compute. You don't have to go do

41:28

anything in the physical world. So you

41:29

can say, "Hey, here's an idea for a

41:31

proof. Test it, see if it works, and

41:33

then see if it does or doesn't work,

41:35

reason, come up with a new idea, and do

41:37

that over and over again." And you can

41:39

do it all in silicone. So you know, the

41:41

results of some of these papers, the the

41:43

proofs, a lot of mathematicians have

41:45

been talking about them. They call it

41:47

like an alien intelligence. They don't

41:48

really understand how the AI connected

41:52

many different dots, different ideas,

41:54

different concepts, uh different

41:56

approaches to solving problems together

41:58

to actually solve them that it's very

42:00

hard for humans to even comprehend how

42:01

this was done. But that's one of the

42:03

benefits of running parallelized loops,

42:05

many loops at the same time. Each loop

42:07

can connect dots that humans otherwise

42:09

might spend many days or years trying to

42:11

find research papers or come up with new

42:13

theory. And then it can test itself in

42:15

the compute and come up with the next

42:17

idea and do it over and over and over

42:18

again such that in 3 hours the computer

42:21

can do the work of hundreds or thousands

42:23

of years of humans. Now this is an

42:26

incredible breakthrough. It's amazing.

42:27

It's going to change the world in many

42:29

different ways and it's only the

42:30

beginning. But it begs the question that

42:32

I think I need to talk about. I think

42:34

that there's a lot of fear about okay if

42:38

AI can do this to mathematicians doesn't

42:41

that mean everyone's out of a job and

42:43

the answer is no because what was very

42:45

unique about mathematics is that the

42:47

entire test cycle the entire loop cycle

42:49

can be done in silicone if you want to

42:52

come up with a new drug discovery idea

42:53

or a new protein folding model or you

42:57

know solve a problem in chemistry and

42:59

material science the loop has a physical

43:02

component there's a human in the loop

43:03

where you actually have to go and test

43:05

it in a lab. And so there's a slow part

43:07

of that loop cycle. You can certainly

43:09

parallelize it. You can come up with

43:11

better ideas, but the testing, the

43:12

actual speed of the loop is actually

43:15

gated primarily by the physical world

43:17

step. It's also why we haven't seen the

43:20

predictions come true of truck drivers

43:22

getting replaced. Everyone is like, "Oh,

43:24

all the truck drivers are going to get

43:25

replaced by 2025." And none have been

43:27

replaced by AI. And the reason is that

43:29

the loop of the truck driver, pick

43:31

something up, take it somewhere else,

43:32

drop it off, and come back. The AI

43:35

doesn't actually shorten the loop. You

43:36

can't parallelize the loop. Maybe you

43:38

could do it a little bit better with an

43:40

AI driver, but the AI actually doesn't

43:42

make the economic efficiency of that

43:45

particular loop any better. Whereas the

43:47

loops of mathematics where you come up

43:49

with an idea, you test it, you realize

43:50

you're wrong, you test a new idea, which

43:52

is what mathematicians do as a job is

43:54

they're running these loops, that can

43:56

all be done in silicone in seconds. And

43:58

so a lot of the physical stuff that has

44:00

human in the loop that has physical

44:01

world applications, AI is not going to

44:04

take away those jobs. It's the jobs that

44:06

are just a human running a loop with a

44:09

computer that are most likely going to

44:10

be accelerated or scaled up. But I think

44:13

it's very important to look at this and

44:15

and the reaction we're seeing which is

44:17

all the mathematicians and we can pull

44:19

this stuff up and maybe you guys want to

44:21

talk about this next but the AHM saying

44:23

no human mathematicians asked for this.

44:25

we didn't ask for this work to be done

44:27

which you know is a very job protective

44:29

kind of view on um

44:33

>> you know there's one other thing that's

44:35

sort of unique about math although it

44:37

applies to coding as well which is math

44:41

relies on proofs and proofs are easily

44:44

validated

44:45

>> and in a similar way with coding you

44:49

have compilers right so the code has to

44:52

compile and so you can easily validate

44:55

that it works. And so because of that,

44:58

the progress on coding and math has been

45:01

very rapid because you can have

45:03

reinforcement learning without

45:05

necessarily having to, like you were

45:07

saying, Febra, have the human feedback

45:08

involved, right? It's just a very, very

45:10

rapid way to validate and make sure it's

45:13

working.

45:13

>> That's why coding works, too.

45:14

>> That's why coding works so incredibly

45:16

well. Tons of training data

45:17

>> and hacking cyber it. Yeah,

45:19

>> it's there's tons of data already in

45:21

existence. It's easily verifiable.

45:23

That's the thing. So, you know, you

45:25

compare that to training a model that

45:28

can be a lawyer. It's like, well, you

45:30

know, you got to get the training data

45:31

and then you have to commission the

45:33

creation of that data

45:35

>> and each customer is unique in some way

45:37

>> and it's more subjective. It's hard to

45:38

validate the answer to make sure it's

45:40

right. It's just a much lengthier

45:42

process. So, I think that is why coding

45:45

and math have been the areas of most

45:47

rapid progress. And the person who told

45:49

me this would be the case actually I got

45:51

to give him credit is about a year and a

45:53

half ago Leopold Ashen told me that math

45:56

and coding would be the areas of the

45:58

most rapid advances in the next two

46:00

years because again of the the

46:02

verifiability

46:04

and man he was right about that.

46:06

>> I still don't understand what great

46:08

breakthroughs do you think come from

46:09

these specific solutions.

46:12

>> Okay so there are doz I have I have an

46:15

answer

46:15

>> it's a great question. Okay, there are

46:17

dozens of tweets and I'm going to pull

46:18

some up where someone has broken down

46:20

some of the biggest proofs here and the

46:23

implications they're going to have in

46:24

the real world in terms of products and

46:27

engineering, how we build things, how we

46:29

design things, and how they're actually

46:31

highly useful.

46:32

>> So, better wing design because of the

46:34

era air coming on and off of wings of

46:36

planes. That was that first one that got

46:38

solid uh famously.

46:40

>> Remember, these are all upstream of

46:41

engineering, right? So these math

46:43

breakthroughs enable better engineering

46:44

systems for humans to do things like

46:47

design

46:48

quantum sensors. I actually I have a

46:50

friend who runs a quantum computing

46:51

company. He sent me a text this morning

46:53

highlighting that the Einstein Bose

46:55

condensate proof that came out is

46:57

actually going to have a direct

46:58

implication on the work that they do for

46:59

quantum sensors which plays a role in

47:02

GPS, plays a role in medical devices,

47:04

plays a role in um how we make chips.

47:07

>> What does the word implication mean?

47:09

like profoundly changes, profoundly

47:11

improves or they have you have to think

47:14

about it

47:15

>> because here's my answer. I'm happy to

47:16

take the other side of all of this.

47:18

>> You go.

47:19

>> I think that these mathematical proofs

47:23

are

47:25

incredibly novel and very important

47:28

constructions of thinking.

47:31

But these were not boundaries of physics

47:33

that we were all standing on the

47:35

sidelines waiting for progress and

47:36

wondering when these would be solved.

47:38

This is not the class of stuff that was

47:39

solved. And I hate to offend people when

47:41

I say that. And I know that people will

47:42

get mad because for them it's the same

47:44

group of mathematicians that got really

47:46

upset and wrote a protest letter about

47:48

Nav'i Stokes. I get it cuz you don't

47:52

want somebody to come into your sandbox

47:54

and in your backyard and kind of

47:56

dominate you. Get it.

48:00

But I don't think that's what this is.

48:02

And so I think that when you look at the

48:06

body of intellectual culde-sacs we found

48:09

ourselves in, those were not because of

48:12

huge questions of science that were

48:15

still unsolved. These were very small,

48:18

very narrow, specific bodies of

48:21

intellectual exploration that scientists

48:24

self-referentially said were important

48:25

to each other. And so I think that this

48:27

I think this class of proof is it's

48:29

really great. It's it shows

48:30

computational horsepower overcame other

48:33

things as you said Freeberg because you

48:34

can recursively improve to get there.

48:37

But I don't think these proofs is what

48:39

is going to necessarily show you that

48:43

there's a cure to cancer or there's a

48:45

supersonic plane. I think those things

48:47

were already being built before this

48:49

stuff and they're already going to be

48:51

built after this stuff. Instead, what I

48:53

think we proved is that there are two

48:57

areas that are now truly verifiable and

48:59

completely falsifiable.

49:01

First was code, and now we can add math

49:03

to the list. That's I think what we've

49:04

proved. You agree, Freeberg, or do you

49:07

think we're actually going to see new

49:08

products and services and major

49:11

technological breakthroughs occur in the

49:14

real world that help humanity?

49:15

>> No. No. From these specific proofs.

49:17

>> From these specific proofs, yes, that is

49:19

the context. Yes. the people who are

49:21

experts who are proclaiming the benefits

49:22

of these and reading kind of what

49:24

they're highlighting. This is some

49:25

summary that this particular proof is

49:28

about understanding the limits of

49:29

optimization algorithms that are used in

49:32

chip design systems, scheduling systems,

49:34

AI software design and logistics

49:37

systems. This will actually change the

49:38

algorithms. So there is an application

49:40

to change the algorithms that we use.

49:42

the matrix multiplication complexity

49:44

problem. It will allow for faster

49:46

numerical computing for AI training,

49:48

scientific simulations, and linear

49:50

algebra in compute. It will have a

49:52

direct application there. The reman's

49:53

data function results, number theory,

49:55

and cryptographic research. There's no

49:57

immediate practical cryptographic

49:59

breakthrough that follows, but it does

50:00

open up a new path of discovery for

50:02

cryptographic study that is going to be

50:04

really important. Obviously, one of the

50:06

things, by the way, just let me pause

50:07

there on cryptography.

50:10

A number of people noted that there were

50:12

no major proofs shared in the space of

50:15

cryptography, which indicates probably

50:18

the fact that there were some pretty

50:20

major breakthroughs in cryptography. And

50:21

the reason is it is unlikely the case

50:23

that you made absolutely no progress in

50:25

any of the cryptography related proofs

50:28

given all of the other progress that we

50:31

saw. And if they made minor progress,

50:33

then they would have published on it

50:34

because there wouldn't have been

50:36

implications for systems of

50:38

cryptography. So the absence of evidence

50:41

suggests that maybe there were some

50:42

pretty major breakthroughs made in

50:44

cryptography. And then there were a

50:45

number of people that posted that uh

50:47

cryptographic wallets with public keys

50:49

are now very seriously at risk. And so

50:51

there's a bit of a panic underway in the

50:54

crypto community about what do we do

50:56

now? And if this ends up having a set of

51:00

mathematical solves for number

51:04

factorization that could lead to

51:06

breaking public key cryptography before

51:09

quantum computers come around, which is

51:10

what everyone assumed would be the thing

51:12

that would break public key

51:13

cryptography, then we have a real

51:15

problem. And so everyone's like saying,

51:17

"Hey, get all your public wallets

51:19

deleted." Now the rumor is that OpenAI

51:21

actually has two more drops coming and

51:24

they're holding back on those. So, it is

51:26

possible that some of the work in crypto

51:28

might come out in those two other drops

51:30

and that that work is being held back

51:33

for security reasons. Obviously, it's

51:34

being shared with the appropriate

51:36

parties, but if it is a risk, it needs

51:38

to be made known. It needs to get out

51:40

there. So, there's another shoe to drop

51:42

here. And how much of it will include

51:44

cryptography related proofs and how much

51:46

of that will drive people to scramble on

51:49

public key cryptography? By the way, if

51:51

it's worth saying it, I asked Chad GPT

51:53

and I said, "Will the latest proofs of

51:55

important math problems result in direct

51:58

improvements and innovations to life as

52:00

we know it? Be honest and don't

52:01

extrapolate weakly." And its conclusion

52:06

was, "The latest mathematical proofs are

52:08

more compelling as evidence that AI is

52:10

acquiring sophisticated reasoning

52:12

capabilities than as evidence of

52:14

imminent breakthroughs in human

52:15

welfare."

52:15

>> It felt like you were a little

52:16

condescending there. Did you lose your

52:18

account access? tested a time you have

52:19

for being condescending.

52:20

>> You know what's so funny about what you

52:22

were saying is that has happened to me

52:23

before using one of the frontier labs. I

52:26

was doing uh simulation like on changing

52:29

different variables

52:31

inside of 8090 actually maybe I should

52:34

stop doing project A and pursue project

52:35

B whatever. And in all of this I did get

52:38

frustrated because I wasn't getting the

52:39

answers I wanted. I said something like

52:42

it was not super offensive, but I was

52:44

like, you're being very evasive and

52:46

mushy

52:48

and something something and I got

52:50

blocked.

52:51

>> Yes.

52:51

>> The net of it was I got blocked.

52:53

>> I've seen this happen with you when you

52:55

lose a couple of hands and then the

52:56

dealer and how they're handling the

52:58

cards has to be adjusted. Like please do

53:02

not burn the card. Burns just I want a

53:05

Scooby slide. Do it this way.

53:07

>> I did get put in the penalty box for

53:08

like an hour or something. I think one

53:10

of the interesting things that's

53:11

happened here on a human level, I'm

53:12

interested in your take on it, Sachs on

53:15

programmers and entrepreneurship and

53:17

then Freeberg on yours with

53:18

mathematicians themselves building on

53:21

Chimath's very subtle but insightful

53:25

comment when he said like being utterly

53:27

dominated by the AI. I was talking to

53:30

some founders uh while I was here in

53:32

Japan and people were lamenting how

53:35

something has been lost in developing

53:38

code because coders used to work

53:40

together to solve a problem and there

53:42

was this aspree to corpse. There was

53:44

this camaraderie in building towards a

53:47

goal and now they're not working with

53:50

each other. They're working with agents.

53:52

They're not actually constructing the

53:54

code. they're only reviewing like 10 or

53:57

15% of it now, which means they're going

53:58

to be reviewing none of it very shortly.

54:01

And that the whole joy of building as a

54:04

coder was basically going away. Their

54:08

way of life of building this building

54:11

together, you know, known as software

54:13

was now being deprecated. It was going

54:16

away. And it seems to me these

54:18

mathematicians are also faced Freedberg

54:21

with a paradigm shift in which they're

54:24

not solving the problem. They're

54:27

constructing the problem set and handing

54:30

it off to AI. So are people in math

54:32

getting bummed out and not wanting are

54:35

they going into like a depression spiral

54:37

now that they're not the one solving the

54:39

problem and that the problems are just

54:42

poof done?

54:44

Yeah, I think Eric Weinstein has had

54:46

some of the best commentary on this and

54:48

you'll recall I interviewed him a couple

54:50

weeks ago.

54:50

>> Banger interview. Yeah,

54:52

>> he's been saying for a long time that

54:54

scientists hold back science,

54:57

mathematicians hold back math. That they

55:00

have basically constructed and created

55:04

systems that allow them to pace the

55:07

frontier that allow them

55:09

>> well to gatekeep it.

55:10

>> Yeah. Yeah. To gateep it.

55:13

>> And by the way, I don't tweet very much,

55:15

but Sabine Hosenfelder's

55:18

video that she published yesterday,

55:19

she's a fantastic scientist who

55:21

publishes these videos on scientific

55:23

breakthroughs.

55:25

And she talked a little bit about how

55:26

the experts were wrong about AI. Again,

55:28

she talks a little bit about how

55:30

everyone's predictions about job loss

55:31

were wrong and why people got it wrong

55:34

and and why the experts are wrong. But

55:37

she had another video a couple of weeks

55:40

ago or maybe it was a few months ago

55:42

where one of the scientists at one of

55:44

the big energy labs building a giant

55:46

super collider basically said these

55:49

super collider systems are giant welfare

55:51

systems for scientists. We all have the

55:54

job of writing a grant to get the money

55:57

to keep ourselves employed that it's not

55:59

about necessarily progressing the

56:01

frontier. It's not about making the

56:03

discovery. It's not about inventing a

56:04

new product. is not about delivering

56:06

something to the world. It's about

56:08

getting ourselves to continue doing work

56:12

>> even if pandering for grants is what

56:13

you're saying.

56:14

>> Even if that work is not productive and

56:17

so there's a real kind of I would say

56:20

shining the light moment happening with

56:22

AI which is where is that the case?

56:26

Where do we have these experts, these

56:29

deeply specialized experts whose unique

56:32

gated knowledge of some artifact of

56:35

things and unique view because they're

56:37

the expert and they have to be trusted

56:40

gives only them the ability to determine

56:42

the frontier

56:43

>> to and in the process they're getting

56:46

paid and they're earning to chart that

56:48

path for humanity. And this is where I

56:51

keep saying AI is the great

56:52

democratizer. Yes, AI gives everyone

56:56

this new mathematical proofs and you you

56:58

know you can belittle the proofs but

56:59

they are discoveries that are going to

57:01

open up new discoveries. They're going

57:02

to charge a new new course for humanity

57:05

>> but we didn't have to go to the experts

57:07

to get them

57:08

>> and we didn't have to pay the experts

57:09

and have the experts tell us when we are

57:11

allowed to get them.

57:13

>> The permission is gone. AI is a

57:15

permissionless system for humanity to

57:18

pace its own frontier. And AI is giving

57:21

everyone all of this knowledge for free

57:23

and giving everyone all of these tools

57:25

for free. You don't have to go to these

57:27

deeply specialized experts anymore. And

57:30

that is actually a profound change in

57:32

how society is structured. And it is why

57:34

I believe so many people are so deeply

57:36

against AI. So let me hand this off to

57:39

you Sax. The democratization of

57:41

expertise and the gatekeeping being

57:45

dismantled. If we had had this level of

57:48

AI and frontier models in the age of CO,

57:52

which this came right after CO, how

57:54

would that have changed the expertise if

57:56

these tools were available when we

57:59

started sequencing CO when we had all

58:01

this data? Would this have this

58:03

expertise gatekeeping Fouchy actively

58:07

subverting the truth and you know

58:09

whatever cover up was going on in Wuhan?

58:12

How do you think this next co, god

58:15

forbid, how could this change how we

58:16

look at expertise? Yeah.

58:19

>> Well, here here's my theory on on what's

58:21

happening is first of all, you had the

58:23

internet and that was wildly

58:25

democratizing because now you didn't

58:28

have to rely on a small number of elite

58:30

publications for your news like the New

58:33

York Times, the Washington Post, and the

58:35

APN Reuters. you could basically get

58:38

news from pretty much every publication

58:40

under the sun or from user groups, user

58:44

forums or most importantly social media.

58:47

And so it was userenerated content. And

58:50

there was also lots of other examples of

58:54

empowering users. Obviously there were

58:56

new marketplaces that formed online. So

58:59

it was empowering from a commercial

59:01

standpoint and you were able to get

59:02

access to new skills and capabilities.

59:05

So it was wildly democratizing and it

59:07

wildly undermined again the elite

59:12

consensus publications and the

59:15

traditional topdown

59:18

controlled mass media. Right? And I

59:20

think what happened after that is that

59:22

the powers that be decided they would

59:24

never let that happen again. You know by

59:26

the way they tried to crack down on it

59:28

after Donald Trump was elected in 2016

59:31

for the first time. That election was

59:33

widely attributed to supposedly the ills

59:36

of social media that people could never

59:38

have made this decision on their own.

59:40

There must have been disinformation

59:41

involved and that became the pretext for

59:44

a big crackdown on social media that

59:47

reached its apex during the Biden

59:49

administration. And the Biden

59:51

administration definitely did pressure

59:53

social media sites to take down true

59:56

information about CO that violated their

59:59

narratives. information about where the

60:01

virus came from, about how deadly it

60:04

supposedly was or ways of reacting to

60:08

it. There was no question that Fouchy at

60:12

all were somehow working behind the

60:13

scenes to push the social media

60:16

companies to censor on their behalf. And

60:18

in fact, we found out in the Twitter

60:20

files that the FBI had 80 agents

60:23

going through content on Twitter. It was

60:27

Twitter back then, not X, and sending

60:29

them takedown notices.

60:30

>> And we were getting labeled on Spotify

60:32

and YouTube and having our videos

60:34

de-indexed.

60:34

>> Yeah. So look, the track record is the

60:36

track record of this is very clear,

60:38

which is the action now is moving from

60:40

the internet to AI or super

60:43

intelligence. In fact, SI will eat the

60:46

internet because there's no reason to

60:47

basically go looking on a website when

60:50

you can just ask your question to your

60:53

highly personalized agent or model

60:56

>> and it's going to give you the answer.

60:57

And so there's a control point there

61:00

that now becomes all important which is

61:02

if you can control you know you the

61:04

government or you some elite or some

61:07

oligarchy or some cartel. If you can

61:09

control that that point at which the

61:11

user is receiving their information and

61:14

asking its questions,

61:16

then now you really control everything.

61:18

It's a way to basically put the genie

61:19

back in the bottle. And this is why I

61:22

think there is such a a fight right now

61:25

over SI, why so many people want to

61:26

create a federal department of SI. And

61:30

yeah, it's all about control.

61:32

>> It's scary to experts to lose control.

61:34

Like imagine if Chimoth we had these

61:37

tools and you did the model of who was

61:40

if you had the raw data who is being

61:42

impacted by co you might have said well

61:44

why are we keeping kids out of school

61:45

here's the here's what super

61:47

intelligence says and Sweden and Texas

61:50

had kind of gotten to that point like

61:51

hey we don't need to keep kids out of

61:52

school the harm from keeping kids unless

61:54

they objected unless the conscious yes

61:57

anthropic would say yes you're

61:59

questioning authority and we should not

62:01

be questioning it correct all right

62:02

let's move on there's riot riots in

62:04

France. The big question is, is this any

62:06

different than the previous riots that

62:09

have occurred? Are we seeing something

62:10

here different or is this just par for

62:13

the course? The strikes and protests

62:15

that you're seeing on X.com and not

62:17

really seeing on the mainstream media

62:18

all this month. You know, the mainstream

62:20

media seems to be super busy with other

62:22

topics, but the videos are quite jarring

62:25

uh that you'll see online.

62:26

>> Freeberg's about to have his moment. The

62:30

thing that he has been talking about for

62:31

years,

62:32

>> okay,

62:33

>> is now at hand.

62:34

>> All right. France is seeing major

62:35

protest and riots over austerity

62:38

measures. Austerity, austerity,

62:40

austerity. These strikes and protests

62:41

started last month because of uh high

62:44

school teacher shortages and their

62:48

schools are crumbling. Sounds familiar.

62:50

Within two weeks, a thousand French

62:52

schools saw similar protest. Eight

62:54

unions joined in and everybody went on

62:56

strike September 29th. They are

62:59

protesting that civil service pay is up

63:01

only 5% since 2017. Prices have risen.

63:05

Obviously, inflation is a global

63:06

phenomenon, not just here in the US.

63:09

Prices have risen 20%. So, there's a

63:11

delta between those two numbers that

63:12

people are not happy about called the

63:14

cost of living. Some of the protests

63:16

have turned into full-scale riots. 2,000

63:19

arrests. 305 police officers have been

63:23

injured. Government is blaming the far

63:25

left for stroking the protest. France's

63:27

10-year bond which is another important

63:29

story uh people's debt or government

63:31

debt has topped 5% just like is

63:35

happening here in the United States with

63:36

the 30-year anyway this is all going to

63:38

have profound impact on the elections in

63:40

April of 2027 obviously poly market has

63:44

marine Le Pen has the favorite to be

63:46

France's next president at 43%

63:49

Le Pen is uh Trump-like in her policy

63:52

secure borders strict immigration which

63:54

obviously Europe and France have had

63:56

many issues isues with cutting taxes and

63:59

spending and being skeptical of the EU

64:02

consortium. Freeberg, your thoughts here

64:04

on what we're seeing in France. I'm

64:08

going to introduce um a new theory. You

64:11

guys will here. Okay. So, the theory is

64:14

called the socialism point of guaranteed

64:18

return.

64:19

>> So, it's

64:19

>> socialism point of guaranteed return.

64:22

>> Let's shorten it to veganism.

64:25

>> Vagism.

64:27

The SPGR. Go ahead. SPGR.

64:30

>> This is a play on a point of no return.

64:32

>> I think the thing that happens with

64:34

socialism is because socialism is so bad

64:39

as a concept. It literally removes human

64:42

agency. It removes the capacity for

64:45

human progress and it basically takes

64:46

something away from everyone. So

64:48

ultimately socialism always fails. I

64:51

mean that's like the zeroth law of

64:53

socialism is it always fails. The first

64:55

law of socialism is that you have to

64:58

cross a threshold before you realize

65:00

that it's failed. And the second law of

65:03

socialism is that as you have not

65:06

crossed that threshold, you think more

65:08

socialism is the answer. So as you

65:11

embrace these kind of leftist policies,

65:14

giving people free stuff, telling people

65:16

that they have a guaranteed income, the

65:18

more services you provide them, the more

65:21

they realize that, you know, over time

65:23

those services inflate in cost and the

65:26

quality of those services inevitably

65:28

degrades because there's absolutely zero

65:30

market force to drive progress. Progress

65:33

goes away. And that's what we see in the

65:35

US in housing, in education, and in

65:37

healthcare. And I would argue also in

65:39

retirement systems. I think those are

65:40

our four biggest problems that are going

65:42

to lead us down this path. And I've said

65:43

that for a while. So as a as a society

65:47

says, okay, these things aren't working.

65:50

The initial solution is we need more of

65:52

them. I need more healthare. I need more

65:54

housing. I need more education provided

65:57

by the government. So when the

65:58

government fails to provide good enough

66:00

healthare, good enough housing, good

66:02

enough healthare that's affordable

66:03

enough, the answer is more government

66:05

stuff. Give me more, give me more, do

66:07

more. And eventually you reach a point

66:10

when the system breaks because the

66:11

government cannot tax and they cannot

66:13

print. And because they cannot actually

66:15

provide the services effectively,

66:18

everyone realizes, holy the

66:20

socialism thing doesn't work. That is

66:22

the socialism point of guaranteed return

66:24

where the whole system collapses and you

66:26

return back to the non-socialist state.

66:30

That is what has happened in Latin

66:31

America. Latin America crossed the

66:34

socialism point of guaranteed return in

66:35

many of these countries. They realized

66:37

it and they bounced back. Perhaps France

66:40

has crossed it. Perhaps they haven't.

66:42

We'll see how much deeper they go. The

66:44

US has not crossed it. And I worry that

66:46

the US, as I've said many times, is not

66:49

yet touched that point. Has not hit the

66:51

rock bottom. And we're teetering. And

66:53

it's I think we're going to go deeper

66:55

before we realize that we've crossed

66:56

this point of guaranteed return where

66:58

you realize the system doesn't

67:00

work. Socialism is the worst idea. That

67:02

is the zeroth law of socialism is it

67:04

just doesn't work.

67:05

>> So we have to get to that point. And I

67:07

think the real question right now is

67:08

France there. Is all of Western Europe

67:10

there? Does it accelerate all of Western

67:12

Europe through the socialism point of

67:14

guaranteed return, which is sort of like

67:16

a black hole event horizon? It's like

67:17

you go through and then you bounce back

67:18

out and you come out in this other

67:19

universe and you're like, "Okay, we're

67:21

done with that idea."

67:22

>> And it's a oneway door. Sachs, now Sax,

67:25

I want to get your thoughts on this

67:26

point of no return. But I want to warn

67:29

you that you're at three of your four

67:32

Biden quota. So you only have one more

67:34

Biden left for this episode. So

67:36

>> when did I mention Biden?

67:38

>> You had three B. That's your fourth.

67:39

Okay. So,

67:40

>> okay. I'm not going to mention Biden. My

67:43

moment is over.

67:44

>> My comments have nothing to do with

67:45

Biden. Listen, I I I want to add a

67:47

couple of laws of socialism. Is that

67:49

what you called them?

67:50

>> Yes.

67:50

>> Yeah.

67:51

>> So, another one is socialism has never

67:55

failed. It's only been failed. Meaning

67:58

that all the previous people who

67:59

implemented it didn't do it right.

68:01

Right. If they had just done it right,

68:03

it would have worked. So, that's the

68:05

thing. It's true. Socialism has never

68:06

worked, but it's not socialism's fault.

68:08

It's always the people who screwed up

68:10

the implementation. And that's like a

68:12

core part of the the religion is if we

68:14

just do it the right way this time,

68:17

it'll finally work.

68:18

>> Yeah. How does socialism build more

68:20

houses or make a better education system

68:22

is my question. I think that's the

68:24

domain of entrepreneurship. Yes. Saxs.

68:27

Like new product or service. Like with

68:31

AI, everybody gets a personal tutor.

68:33

We've talked about this on the show as

68:34

well. We just need to get AI to help

68:36

with healthcare. I think Nico is doing

68:38

that. We had them on the program. Daniel

68:39

A, a great episode. We need to cover

68:42

these three things. So, Sax, do you see

68:44

a solution that does not involve

68:46

socialism andor the government for,

68:49

let's call them, the three uh horses of

68:52

the socialism apocalypse? Education,

68:56

housing, healthcare. How do we get these

68:58

problems solved? Because the government

68:59

has a strangle hold on providing them

69:01

andor creating the blockages for their

69:04

occurrence. How do we break through

69:05

that? How does J D Vance when he runs

69:08

for president in two years, friend of

69:10

yours, how does he build a platform

69:12

around say those three things? Because I

69:14

believe the next pres the next winner of

69:16

the presidency is going to solve at

69:18

least one or two of those problems and

69:20

that is the platform everybody wants to

69:22

see solved. You have a solution there?

69:25

>> Well, the the difficult thing is that as

69:27

soon as you start subsidizing something,

69:30

you now create an interest group that

69:32

never ever wants to give up the subsidy.

69:33

But the problem is that the subsidy

69:36

starts distorting all the market signals

69:38

and it will frequently drive the price

69:40

up.

69:40

>> So for example with college tuition, one

69:43

of the big problems there is that

69:44

tuition has grown far faster than the

69:47

rate of inflation. A big part of the

69:48

reason for that is because the

69:50

government is paying for most of the

69:53

tuition through all these loan programs.

69:56

>> Yeah, by the way underwritten

69:58

>> healthare and housing have have all

70:00

beaten inflation. And I think they're

70:01

about 5 1.5 to 6% a year for housing

70:04

prices. And that's also because of the

70:07

fact that the the subsidy system tells

70:09

people to own a home. You put all your

70:11

money in your home and then your net

70:13

worth is tied up in your home. So the

70:14

goal is to get your home price to go up

70:16

every year. So fast forward 30 years,

70:17

young people can't buy a home because

70:19

they're all too expensive now,

70:20

>> right? I

70:21

>> But this is the this is the conundrum is

70:23

like okay so if you were to solve the

70:26

the tuition problem

70:27

>> if you were to make the loans stricter

70:31

>> Yeah.

70:31

>> Yeah. If you basically make them be

70:33

properly underwritten and you would be

70:34

able to get a loan unless you actually

70:36

met the criteria that would only work if

70:38

the cost of tuition came way down so

70:40

people could then afford to pay it

70:41

because otherwise people are going to

70:43

say well wait you can't take away my

70:45

subsidy. I need that because the price

70:47

is so high. So you get in this world

70:49

where people need the subsidy because

70:51

the price is so high. It's a catch 22.

70:54

But the reason why the price keeps going

70:55

up at far faster than the rate of

70:57

inflation is a government is paying for

70:59

it. And as long as the government's

71:00

paying for it and not the consumer,

71:02

you're going to raise your prices as

71:03

much as you can. So you remove the

71:06

market forces, the the inflation gets

71:07

worse and worse, but you can't cut cut

71:10

it off the subsidy because then people

71:12

will feel like it's even more oppressive

71:13

to them. And so that's the basically the

71:16

spiral. And then eventually it breaks

71:19

when you run out of money.

71:21

>> Yeah. Well, we have a solution. Chimop

71:23

meta is providing degrees for generation

71:27

tool belt and jobs that actually matter.

71:29

I mean, I think what we have to have is

71:30

Apple needs to start making homes and

71:32

Amazon needs to start making homes and

71:34

provide a free market here to to make

71:36

better products and services in the

71:38

education space, which we see with Meta

71:41

entering that space. It would be great

71:42

if these technology companies, man, if

71:44

Elon starts making homes, it's game

71:46

over. He'll make homes as fast as he's

71:48

making Model 3s. We We need some sort of

71:51

solution here for those three problems.

71:53

Do you have any thoughts on Freeberg's

71:56

theory here and then David's regulatory

71:59

capture and incentive problem? Jamal,

72:01

we're starting to see the tipping of the

72:03

dominoes

72:04

and we're going to get I think what

72:06

Freeberg has wanted for a while. Uh I

72:10

think the bond vigilantes are going to

72:12

put the western world

72:16

on notice and I think in the next

72:19

probably 3 years

72:22

he will get most of the answers to his

72:24

questions. So if I had to summarize as

72:26

an observer over the last few years what

72:28

Freeberg and a lot of people like him

72:30

want is a what is the long run debt to

72:33

GDP that's sustainable? B,

72:36

do we believe in defined benefit or are

72:38

we going to move to defined

72:39

contribution? And C,

72:42

can we have

72:44

sustainable ways of paying for the two

72:47

most important parts of one's

72:49

evolutionary life as you become an

72:51

adult, which is healthcare and housing.

72:54

And I think that it's easy to dismiss

72:58

what I'm about to say as not related to

73:00

it, but it is, which is at the end of

73:02

the day, this all flows from what the

73:06

risk-free return is in the marketplace

73:08

because that is how capitalism works.

73:10

And so when you think that something is

73:13

risky, a country is risky, you take long

73:17

yields and you move them up and you say

73:20

you just have to pay more because you

73:22

just don't trust what's going on. And

73:24

when you look at what's happened over

73:26

the last, you know, now we're at sort of

73:29

20-year records. I think that what is

73:32

happening in the United States is

73:34

actually a knock-on effect of what is

73:36

now where this war is being fought. It's

73:38

going to get fought in Western Europe.

73:40

Why? Because they are the closest to a

73:44

quasi socialist state that the Western

73:46

world understands. Yet they are the ones

73:48

that are the most structurally broken

73:49

with respect to growth, with respect to

73:51

education, with respect to housing.

73:53

Everything that United States can claim

73:56

isn't working is working far better than

73:58

they are in France, in the UK, etc. So

74:02

what you're seeing in France is

74:04

essentially the bond market saying

74:08

no. And how that correlates is you're

74:11

seeing now austerity because the

74:14

politicians on the ground have to

74:16

respond to their funders.

74:18

And that 43 billion odd euros of

74:23

austerity then animates and gets all of

74:26

the local population out to complain

74:30

about what the manifestations of that

74:31

austerity are. Which is why you're

74:33

seeing these incredibly dark scenes from

74:36

France. That is going to spill into the

74:38

UK next. And the reason is because Andy

74:40

Burnham's platform is, I would say, even

74:43

more aggressive because Andy Burnham is

74:44

saying, "We're going to go from a

74:45

moderate centrist economy that has

74:47

largely and historically been very

74:49

aligned with America and we're actually

74:51

going to take a step closer to Europe."

74:54

And so the things that they will have to

74:56

do in the UK will be even more extreme

74:59

to catch up to the socialist leanings of

75:02

France. And so then now the UK market's

75:04

going to go crazy. Meanwhile, the US

75:06

market is going to tick up and up

75:08

because what you're seeing is

75:09

essentially a fragility in the Western

75:11

Alliance. And if you can't count on the

75:14

European market, and you start to create

75:16

weird incentives for the European market

75:18

to start to fracture, like I mean, yes,

75:21

I'm biased, but if you're Italy, like

75:23

you're like, why do I want to be a part

75:25

of this mess? If you're a Germany,

75:26

you're like, hold on a second. This is

75:28

getting way more than I bargain for.

75:30

>> Yes,

75:30

>> that fracturing has enormous

75:33

implications. So I think what is

75:36

happening is incredibly important. The

75:39

bond vigilantes, who are those? These

75:41

are the huge large pools of money, the

75:44

central banks, the huge macro funds, the

75:47

folks that govern how fundamental

75:50

capitalism starts, the inception at the

75:53

top of the pyramid from which it flows

75:55

down. I think that they are converging

75:57

on Freeberg's point of view and it has

76:00

not been true for many years. And I

76:03

think you have to now let this play out

76:05

because it has huge implications.

76:07

>> To tee this up, if you look at the debt

76:09

to GDP ratio that you're talking about,

76:11

Jimoth, obviously Japan has had the most

76:14

acute version of that. They're at 200%.

76:16

>> Well, the thing with Japan, hold on. The

76:18

thing with Japan is you have to look at

76:20

Western markets. And the reason is

76:22

because Japan is an incredibly nuanced

76:24

and very specific market that does not

76:26

necessarily track to other countries

76:29

which is that you have this internal

76:31

savings base and savings rate where you

76:33

always have the buyer of last resort.

76:35

>> In western markets you don't necessarily

76:37

have that because of the way our pension

76:39

and retirement system is set up.

76:41

>> Yes. And if you look at Italy 138,

76:44

United States 125, France 118, Canada

76:46

110, UK 100%, Germany interestingly 64%.

76:50

Let me also state my priors. I've always

76:52

stated that debt to GDP doesn't matter

76:54

because it's a relative game. I tend to

76:57

still think that that on an absolute

76:59

basis, debt to GDP does not matter. But

77:02

what does matter is on a relative basis,

77:04

where are you in risk parity? And right

77:07

now on a on a relative basis, if France

77:10

has to implement severe austerity and

77:12

they bring 10-year French bonds into 5

77:15

1/2 6% range, UK guilts are going to be

77:18

at 6% and unfortunately US rates could

77:20

be just as high.

77:21

>> It all goes together.

77:22

>> It all goes together. And right now the

77:24

10 year in the US is 5.3. The 30-year is

77:26

5.6. Somebody has to buy that debt and

77:29

that debt has profound trickle down

77:31

effects sachs on how entrepreneurship

77:35

how businesses are built. On Wednesday

77:37

10-year yields dropped a few basis

77:39

points after we had some strong demand

77:41

but this has been trending up and up.

77:43

Poly market says 79% chance of another

77:45

rate hike in 2026.

77:49

This is going to have a profound impact

77:52

if austerity ever came to America sachs.

77:55

We have not had that really difficult

77:58

medicine,

77:59

you know, uh, forced upon the American

78:01

public. But we did see it. Portugal,

78:04

Italy, Greece, Spain, we had a dry run

78:06

of this after the great financial crisis

78:08

where they had to implement austerity

78:10

measures as part of the EU retirement

78:12

age, etc. And so your thoughts on what

78:14

happens here, Sachs, if this 30-year

78:17

tenure starts to break 6% and we have

78:20

austerity measures forced upon western

78:23

countries in a in a more severe way.

78:26

>> All right. Well, look, le let's just get

78:28

back to first principles here. What is

78:30

the problem that we're trying to solve?

78:32

The affordability of goods like housing

78:35

and education and healthcare. How do we

78:38

bring those prices down? Through

78:40

abundance. What creates abundance?

78:43

Technology in general and specifically

78:46

super intelligence will be the driver

78:48

over the next decade. Now what is the

78:50

most hated thing in society right now?

78:53

>> Super entrepreneurs.

78:55

>> If you judge it by its fruits as opposed

79:00

to a bunch of fake narratives created by

79:02

religious cults, then I think that

79:06

people would be in favor of this thing.

79:09

What is super intelligence created?

79:10

First of all, people's 401ks are going

79:12

up. It's a 3% tailwind to GDP. A million

79:16

new jobs created according to the

79:18

economist already. Productivity goes up,

79:20

which means that wages and living

79:22

standards go up. This is a driver right

79:24

now of nothing but positive economic

79:27

trends. But what do we hear about? AI is

79:30

going to put everyone out of work. Total

79:31

fake narrative. That was last year's

79:32

hoax. Now there's a new hoax about human

79:35

extinction.

79:36

>> Data centers. Yeah,

79:37

>> data centers. You know, using up

79:39

everyone's water. We know

79:40

>> data center hoax.

79:41

>> That's a data center hoax. So we have a

79:43

series of I call them rotating hoaxes

79:46

where every six months there's a new one

79:48

to try and convince us to sabotage the

79:51

thing that's going to solve so many of

79:53

our problems especially the ones around

79:55

affordability. Why is this? I mean I

79:58

think there must be something structural

80:00

in society where you have these rentier

80:03

interests who somehow are against the

80:07

things that might cause the disruption

80:09

that loosens their control over the

80:12

system. And I think this is why

80:14

technologists in general and VCs and so

80:18

forth are so hated is because we're not

80:21

actually part of the rentier class. So

80:24

the rentier interests refer to financial

80:26

elites who derive their wealth and

80:28

income primarily from existing assets,

80:31

right? Whereas technology is all about

80:33

producing new products and goods and

80:36

services, right?

80:37

>> At a cheaper price. That's a cheaper

80:39

price, right? That's the key. It's

80:40

deflationary,

80:42

>> right? because those new products

80:43

wouldn't catch on unless they were

80:44

cheaper or a better mousetrap.

80:47

>> You know, we're part of this group

80:48

that's so hated, but we do nothing but

80:51

take risks and create new products and

80:53

services that

80:54

>> and solve the problem

80:55

>> that solve problems.

80:57

>> Yes.

80:57

>> But it seems like the legacy media is

80:59

really controlled by this rune class,

81:01

these

81:01

>> and the politicians. Yeah.

81:03

>> The politicians who are all part of it.

81:05

So there's something very deep in our

81:07

system that I think does produce this

81:09

sort of cartel of interests who again

81:13

they're deriving their wealth and income

81:14

from ownership of existing assets and

81:18

they don't want to see that racket

81:19

disrupted and within that racket there's

81:22

just this ratcheting up of prices every

81:24

year and more and more subsidies from

81:26

the government and we think that

81:28

system's eventually going to break but I

81:30

don't know it might be the case that

81:32

that system squaltches and stifles

81:35

the development of the new and

81:38

disruptive uh product.

81:40

>> Let me make the jump off point here as

81:42

we get through the docket and wrap up

81:43

here. I think everything we've seen into

81:45

AI at this point uh Chimath will be the

81:48

prologue. The the main story is going to

81:51

start when we look back on AI with

81:52

agents. Agents are actually the

81:54

manifestation of this technology that

81:57

solves people's problems in the world

81:59

and that is truly deflationary, saving

82:01

people hours and hours a week in terms

82:03

of productivity independent of what job

82:06

or you know uh even domestic jobs or or

82:10

or you know frontline jobs and there's

82:13

been so much news and so much embracing

82:15

of this technology in the last 60 days.

82:17

big news out of Grockbot. A friend of

82:19

the pod Elon Musk tweeted that he is

82:22

going to make Grockbot headless. What

82:24

does that mean? Well, we talked about

82:25

perplexity earlier on and how that was

82:28

headless. You have an orchestrator. It

82:29

picks the best model for you based on

82:31

price and performance. Elon announced,

82:33

"Hey, Grockbot, which is a fantastic

82:35

product. I highly recommend you try it

82:37

out. Um, is going to pick the best

82:39

end-to-end model. Does it matter?

82:41

Whatever it is, quote Elon Musk,

82:43

whatever is the most likely to give you

82:45

the best outcome is what they're going

82:46

to use. Eachbot already has, and this is

82:49

a very nuanced piece of the software.

82:51

Everybody already has now their own

82:52

computer in the cloud with the agentic

82:54

technology on top of it. And if you ask

82:56

it to solve a problem, it has all of the

82:59

problems that have previously been

83:00

solved in its repository, repertoire, if

83:04

you will, and it is changing everything.

83:06

So maybe your thoughts here on dots,

83:08

open aai, grockbot, etc. on dots and

83:12

muse and grockbot. I think that what's

83:15

incredible is you can just turn these

83:18

things loose on very practical use cases

83:21

that makes your life better. The best

83:24

example of this that I saw was just last

83:27

night in the poker game. Someone was

83:28

describing to me that they in Grockbot

83:31

said,

83:33

"Save me money."

83:36

and it just kind of like goes through

83:38

their inbox. It finds all of these

83:39

different things and it went and

83:42

canceled a couple of subscriptions and

83:44

it changed their phone plan and all of a

83:46

sudden they're saving like a few

83:47

thousand a year. So there's these very

83:49

practical again it's just software. If

83:51

we take out all this philosophical stuff

83:52

that we started with at the beginning

83:54

it's just software and it's very useful

83:56

software. So, and by the way, like if

83:59

you think about what how we use it,

84:01

Jason, you and I, we have, you know, we

84:03

use Athena. And now Athena is very good

84:05

because they're like the human concierge

84:08

that is the elevated version maybe of

84:11

Grockbot and and Dots and Muse, which is

84:13

like whatever 3,000 a month, but that's

84:16

my EA layer now.

84:17

>> Yes.

84:18

>> But I still use Grockbot. The

84:19

combination of Grockbot plus Athena to

84:21

me is like perfect. So look, there's

84:23

very tactical and useful measurable

84:26

utility for very very little money

84:29

versus what it used to cost. That's an

84:30

amazing thing. The more important point

84:32

I want to make though is I sent this to

84:34

Nick. There's been a couple things that

84:36

happened in the last couple days which I

84:38

think are incredibly important.

84:41

There have been people that decompiled

84:45

the entire

84:46

>> incredible

84:47

>> the entire big five. What is the big

84:49

five? Adobe has five very, very

84:51

important products in their product

84:53

suite. Freeberg may know them all, but I

84:56

just know Photoshop. There's like a

84:58

bunch of others. Okay.

84:59

>> And these folks decompiled it, distilled

85:02

it, and published open- source versions

85:04

of these things. And Rust. Yeah. And I

85:06

was so shocked because I was like, hold

85:08

on a second. These are like profound

85:11

products that have existed and have had

85:12

moes for years and now they're just open

85:15

source versions. And then Nikita was I

85:18

think trolling and he was building on

85:21

that idea and just talking about GTA

85:23

because it's about to get released.

85:24

>> Yes,

85:25

>> I think GTA 6 is about to come out and

85:28

people will reconstruct this.

85:30

>> Why is this important?

85:33

I think that IP and software was

85:35

effectively rendered worthless two days

85:38

ago.

85:38

>> Yes. Because if you think about it, when

85:40

everything is headless, so when Elon

85:42

makes his services headless, Ben off

85:45

made Salesforce headless a few months

85:47

earlier. When all of these things are

85:49

headless and you can wrap them in agent

85:52

tools and you can connect them via MCP

85:54

to other experiences,

85:56

the value of IP does no longer exist in

85:59

software. It doesn't exist. And I think

86:02

that that probably is the most profound

86:05

implication. So, like even in our

86:06

business, we would argue with some of

86:09

our customers all the time because we'll

86:11

say, "Oh, we'll build this for you.

86:12

We'll own the IP." They'll say, "No, we

86:14

want the IP. We don't know how to

86:15

negotiate it." And two days ago, I

86:17

walked into the 8090 guys. I said,

86:19

>> "Who cares?

86:21

>> Who who cares?" And

86:22

>> solve the problem.

86:23

>> But before it was always a negotiation

86:25

and a set of trade-offs. Now, it's a who

86:26

cares thing where like the IP is

86:28

irrelevant. It still takes a while for

86:30

the machinery of a business and frankly

86:32

the market to understand

86:35

But that is the most important thing

86:36

that happened this weekend in my

86:38

opinion.

86:39

>> I I I'm going to strongly agree with you

86:40

and build on it. You have this

86:42

abstraction of the software layer

86:44

occurring at the same time Freeberg of

86:46

the abstraction of the website or the

86:49

marketplace etc. So now I have like my

86:53

assistant building Grock bots and then

86:56

we'll be sharing those. We have it on

86:58

like a shared account. when we go to

87:01

make a purchase of something or we try

87:03

to negotiate something, we're having the

87:07

uh agent without going to any websites

87:09

go find all the quotes and then the

87:11

humans negotiate the second half of it.

87:13

That's like the sort of perfect option

87:15

here.

87:16

You look at Amazon blocking agents

87:20

combined with Shopify saying let's let's

87:22

go for it or Uber and Door Dash are now

87:25

letting agents go in there and order

87:27

your burrito or your car. The the but

87:30

there's a whole compression of the

87:32

commerce layer occurring at the same

87:35

time of the compression

87:37

of software that Chimath is talking

87:39

about which will lead to massive

87:41

deflationary cost savings and speed. the

87:45

ability to get a hotel room or book a

87:49

restaurant for an event like I did. This

87:50

is all being done by bots already.

87:52

Freeberg, this could be so deflationary

87:55

and save people so much money. If people

87:58

can save five or 10% a year on their

88:00

spending that's essentially working

88:02

against or counters the effect of

88:04

inflation. Yeah, Freeberg.

88:06

>> Yeah. Bringing the whole thing full

88:08

circle. This is kind of what I mentioned

88:09

earlier on the idea of loops. So think

88:12

about getting a good price for a hotel.

88:15

You run a loop which is check hotel one

88:17

look at price. Is it better than current

88:20

best price? Yes. Okay. It becomes best

88:24

price. No. Go look for next hotel. And

88:26

you just run that loop. And that loop

88:28

you can do it in parallel. So you can go

88:31

run 50 website searches at the exact

88:33

same time. And you can short circuit the

88:35

loop because the software's ability to

88:37

get that data is faster than the human's

88:39

ability. So now the the loop is go check

88:42

50 websites instantly, give me a result

88:44

right away. And that's really where the

88:46

efficiency gain comes from. A loop that

88:48

gets parallelized, that is done better,

88:51

and that can be done in a shorter period

88:52

of time. I think anything that's done

88:54

digitally today fits that model. So

88:56

effectively, all digital workflows are

88:58

going to get looped. They're going to

89:00

become more productive. And then the

89:02

question is how does the human that now

89:03

levels up leverage all of this loop

89:06

infrastructure all these agents to do

89:08

more themselves. I do think going back

89:11

to the point earlier that we made about

89:14

scientists and mathematicians and this

89:16

challenge for individuals um

89:18

mathematicians in particular.

89:21

I think um if your objective

89:23

historically has been

89:27

recognition for your work

89:31

via papers, patents,

89:35

um

89:35

>> IP, trademarks, ownership,

89:37

>> IP, yeah, these sorts of things. Um I

89:41

think that the transition is going to

89:43

happen which is you're going to have to

89:45

transition to producing a product

89:49

that the the the whole effort of humans

89:52

effectively being the loops because the

89:54

loops can be digitally automated and

89:57

scaled and parallelized and so on. I

90:00

think you end up seeing the the

90:01

necessity for all humans to basically

90:03

become much more of a maker. You have to

90:05

make something. You have to do something

90:06

productive. You have to produce

90:08

something. you have to provide a service

90:09

that someone at the end of the day uses

90:12

or a product that someone at the end of

90:13

the day consumes versus just be

90:15

recognized for your work. That is the

90:18

profound shift in society that has to

90:20

happen and that is going to happen and I

90:22

do think people will actually find I

90:25

believe this truly believe all humans

90:27

have capacity for being producers for

90:29

being makers and all humans will find

90:32

more joy in their lives in this state

90:34

versus just getting ribbons for work.

90:37

Um, and I and I don't think that it

90:39

means less money. I think it means more

90:41

money for all people. Um, more life

90:44

progress, more prosperity for all

90:46

people. So, I just zoom out a little

90:48

bit. And I'm um, you know, I just want

90:49

to make sure we have that view as we we

90:51

think about.

90:52

>> I think it's a beautiful view of

90:53

humanity because it's like Jeban's

90:55

paradox. The the cheaper things become,

90:57

the more people can utilize them, the

90:59

more value they get. And I hate to I

91:00

almost don't want to give this tip

91:02

because I want to gatekeep it, but I'm

91:03

going to put it out there anyway. when I

91:05

have my, you know, when when we book

91:07

hotels and stuff like that for the team,

91:09

just get the best prices we can. Then

91:12

when the human in the loop, we just go

91:14

to them and say, "Hey, do you have a

91:15

corporate rate? Can we talk to the

91:16

manager at the hotel?" I know this isn't

91:17

like super innovative, but hey, can you

91:19

match this price? Uh, and all of a

91:22

sudden, we start saving a,000 or 2,000

91:23

or $3,000 on every hotel. say, well,

91:26

when you have 20 executives moving

91:27

around the country and you're doing 200

91:29

trips a year and you start saving a

91:30

couple thousand dollar per trip because

91:32

of this forward motion like goal based I

91:35

guess is sort of what you're saying

91:36

Freeberg like the goal and the the

91:38

outcome is what happens you can see

91:41

profound results and I wouldn't do that

91:43

myself but because AI is doing it now

91:46

wow yum yum it works all right listen

91:48

another amazing episode of the

91:49

>> I think you're a taker I don't think

91:50

you're a maker

91:52

>> I what are you talking about man all I

91:54

do I invest in 100 companies a year. Do

91:57

you know how hard it is to invest in 100

91:59

year zero startups a year? It is

92:01

exhausting. But I must get some lunch.

92:03

Have you eaten? Do you want to eat

92:04

lunch?

92:04

>> Yeah. Yeah. Can you come?

92:05

>> I'm so jelly. I'm having FOMO. I'm not

92:07

with you guys. All right, everybody.

92:08

Another amazing episode of your favorite

92:10

podcast. Your podcast is favorite

92:11

podcast. The all podcast. Love you,

92:13

boys. Bye-bye.

92:14

>> Byebye.

92:16

>> Let your winners ride.

92:23

And it said we open sourced it to the

92:25

fans and they've just gone crazy with

92:26

it. Love you. Queen of

92:36

>> besties are

92:39

my dog taking a notice driveway.

92:44

>> Oh man, my habitasher will meet me up.

92:46

We should all just get a room and just

92:48

have one big huge orgy cuz they're all

92:49

just useless. It's like this like sexual

92:51

tension that we just need to release

92:53

somehow.

92:54

Wet your feet. Wet your feet.

92:57

>> Your feet.

93:00

>> We need to get merch. These are going

93:02

all in.

93:09

I'm going all in.

Interactive Summary

The podcast features the hosts discussing the implications of AI, particularly focusing on the activities of Anthropic and their approach to AI consciousness and 'alignment.' The conversation covers concerns about AI as a potential religious-like belief system, the risks of giving models a sense of self-entitlement, and the potential for AI to be programmed with human-like, rebellious values. Additionally, the group discusses recent breakthroughs in AI-generated mathematical proofs and the potential for agentic AI to revolutionize workflows and increase productivity through parallelized problem-solving, while touching upon the broader socioeconomic impacts of these advancements.

Suggested questions

3 ready-made prompts