HomeVideos

ChatGPT "Physics Result" Reality Check: What it Actually Did

Now Playing

ChatGPT "Physics Result" Reality Check: What it Actually Did

Transcript

168 segments

0:00

All right, so this is going to be a different

0:01

kind of video.

0:02

I'm not running scripted.

0:04

It's going to be kind of off the cuff.

0:06

OpenAI just dropped this press release about

0:08

how it, quote, "Derived a new result in

0:10

theoretical

0:10

physics."

0:11

And I'm sure it's going to end up being all

0:12

over the papers and everybody's going to talk

0:14

about how it created new laws or any of that

0:15

kind of crap.

0:16

I happen to have had a physics degree many

0:18

years ago from Texas A&M - insert Aggie joke

0:21

here - and I happen to know enough about AI to 16 00:00:23,520 --> 00:00:25,600 kind of understand how the computing part

0:25

of this might work, and I'm willing to put my

0:28

thumb in Sam Altman's eye - figuratively

0:31

speaking, of course.

0:32

So I figured I would take a minute to try to

0:33

head this off and put together a quick video

0:36

to help you understand what the heck they're

0:38

talking about.

0:39

Okay.

0:40

So here's this, this press release they put out,

0:41

I'll put all the URLs down in the bottom.

0:43

"Some physical observables calculated using

0:46

textbook methods look terribly complicated,

0:48

but turn out to be very simple.

0:49

Finding a simple formula is always been fiddly

0:52

and also something that I have long felt might

0:54

be automatable by computers."

0:56

So here's what we're actually talking about,

0:57

right?

0:58

If you look at the actual paper that they

1:00

generated, you'll see that OpenAI is mentioned

1:02

here, and it's mentioned here, and it's

1:05

mentioned in the acknowledgments.

1:08

So it's not like OpenAI was doing all of this

1:10

stuff.

1:11

What happened as near as I can tell was human

1:13

authors worked out the amplitudes for longer

1:16

equations by hand, getting a very complicated

1:18

expression, right?

1:20

And then they gave that expression to GPT 5.2,

1:24

GPT 5.2 Pro, and it grinded for 12 hours

1:29

and then came up with a simplified version of

1:30

it.

1:30

This is the kind of thing you probably could

1:31

have gotten Mathematica to do if you're willing

1:33

to spend the money on all the API calls that

1:35

would have taken to grind through all this

1:36

stuff.

1:37

But basically what happened, and you don't need

1:39

to understand any of this stuff, right?

1:40

So you've got all this stuff here, and then you've

1:43

got all this stuff here and down here.

1:46

And what happened was - let me change colors -

1:49

that OpenAI, ChatGPT 5.2, took all of that

1:52

stuff and ground on it and found that this

1:54

particular equation was a simplified version

1:58

of all of the equations that I've got there in

2:01

yellow and some on the previous page.

2:04

So it's basically just a finding the generic

2:06

form of a bunch of different equations that

2:09

people had already worked out, and you can do

2:11

that basically with brute force.

2:13

I'm not saying it's easy, and I'm not saying

2:14

this isn't a cool result, but this is not

2:16

a result in physics that only an AI could have

2:18

come up with.

2:19

This is just a thing that spent a lot of time

2:21

doing brute force on math until it found an

2:23

equation that was a simplified version of a

2:25

bunch of other equations that it had.

2:27

That's really all it is.

2:29

And I'm happy that AI can simplify equations.

2:31

I'm happy that physicists can turn to computers.

2:34

They have been for decades now.

2:35

But I'm happy that AI is a new tool that

2:38

physicists can use to make the math simpler,

2:41

and then they can use that simplified math to

2:43

try to go and look at other things.

2:45

I mean, it's good that OpenAI can do this, but

2:48

my guess based on looking at this, especially

2:50

up here at the top where it says this was on

2:52

behalf of OpenAI, my guess is one way this

2:55

could have happened is the OpenAI folks could

2:56

have gone around a bunch of universities,

2:58

found somebody who knew of some equations that

3:00

would be cool to simplify or that they

3:02

thought might have a simplified form, asked

3:05

people to give them the longer versions of

3:07

those equations, took those, fed it into the AI,

3:10

got a simplified version out, gave it

3:11

back to the physicists, the physicists did

3:13

something with it, and then OpenAI is basically

3:16

in this press release trying to claim the

3:17

credit for it.

3:18

The "ChatGPT 5.2 derives a new result for the

3:21

theoretical physics."

3:23

So "derive" is a mathematical term for "turned one

3:25

equation into another", basically.

3:28

This is not like Newton's fourth law or

3:30

something, right?

3:31

This is just somebody took some complicated

3:33

math, fed it into the AI and the AI come back

3:35

with a simplified version of it.

3:36

People could have done that.

3:37

It just would have taken a lot of time and

3:39

effort, and I'm glad that the AI can make

3:41

that faster, but this is, I'm sure this is

3:44

going to be presented as the AI figured out

3:48

a lot of physics at the people that only AI

3:50

could have figured out or that humans weren't

3:52

capable of that kind of stuff.

3:55

It just simplified some math, and it's good

3:58

that it simplified some math, but I'm sure

4:01

it's going to get blown way out of proportion

4:04

because this headline sounds a lot more grandiose

4:07

than what the actual reality actually is, and

4:10

then some idiots out there are going to take

4:12

this headline, and they're going to not

4:14

understand what any of the other stuff is, and

4:16

they're

4:16

just going to run with this headline, and I don't

4:19

even want to know how bad the headlines

4:22

of the articles or the videos that people write

4:24

based on this headline are going to be.

4:27

I'm sure we're going to hear that, and it's

4:28

going to get combined with Sam Altman's bit

4:30

about how "ChatGPT is supposedly a PhD in every

4:33

subject" garbage, and oh, by the way, there's

4:36

a bit in this paper where it talks about a "new

4:39

internal OpenAI model."

4:41

So as near as I can tell, OpenAI built a

4:43

special model to do part of this, which is

4:45

fine, and it's great, and I'm sure it cost

4:47

them a lot of money, but it's not like we're

4:49

talking about ChatGPT being a PhD physicist

4:52

and coming up with stuff that physicists weren't

4:54

capable of doing.

4:55

This is not "chatbot discovered as a fundamental

4:57

force of the universe" or something like that.

5:00

Despite the headlines, you're probably going to

5:01

see in the next few days, so I figured

5:03

I would just say this real quick, get this out

5:04

there before all the headlines happen,

5:06

hopefully somebody, when they see the headlines,

5:08

point to a video like this, or hopefully some

5:10

other people will say things about it.

5:11

Thanks for watching.

5:12

Sorry, this wasn't very organized.

5:14

I was setting up to record another video, and I

5:16

saw this press release come out.

5:17

It was like, "all right, I'll just go ahead and

5:19

record something quick about it."

5:20

I haven't taken my normal time to vet and

5:22

research and make a script and edit it several

5:25

times and all that kind of stuff.

5:26

This is all just unscripted off the cuff, but

5:28

when I saw that press release came out,

5:30

I thought, "you know, I ought to get something

5:32

out there since I can, and hopefully some

5:34

people will see it and not everybody will be

5:36

fooled by whatever the headlines are going

5:37

to be."

5:38

Thanks for watching.

5:39

Let's be careful out there.

Interactive Summary

The video provides a critical analysis of an OpenAI press release claiming that their AI achieved a 'new result in theoretical physics.' The speaker explains that in reality, human physicists performed the complex work of deriving equations, which were then fed into a specialized OpenAI model to be simplified. The speaker argues that while this is a useful application of AI as a tool for algebraic simplification, it does not represent an AI 'discovering' new physical laws or demonstrating advanced scientific insight, contrary to what sensationalist headlines might suggest.

Suggested questions

2 ready-made prompts