HomeVideos

Casey Destroys Optimization Myths | TheStandup

Now Playing

Casey Destroys Optimization Myths | TheStandup

Transcript

898 segments

0:00

If you guys do not know,

0:01

chat, chat, everybody in here, Casey,

0:05

trash,

0:06

the startup is going through a bit of a

0:08

change, all right? There's been some

0:09

investment interest.

0:11

And we are now raising a series A like

0:13

any proper multi-billion dollar unicorn

0:16

startup.

0:17

>> Cool.

0:17

>> And so therefore

0:17

>> Decacorn.

0:19

>> Decacorn, really? We're a decacorn? And

0:20

so

0:21

>> Yeah.

0:21

>> I don't know what that means yet, but

0:23

someone told me it's like that's the

0:24

best kind of unicorn. I think it means

0:26

it has multiple horns. I don't know what

0:27

it means.

0:28

>> 10 10 horns.

0:29

>> 10 horns.

0:30

>> 10 horns.

0:31

>> Okay, nice. So it's kind of like the the

0:33

beast in the Bible, the devil, right?

0:35

Very cool.

0:36

>> have 10, does it?

0:38

>> Yeah, yeah, yeah, yeah, yeah, yeah,

0:39

yeah, yeah, yeah, yeah, yeah.

0:42

Uh anyways, I'm sorry.

0:43

>> Fact checkers? Fact checkers?

0:44

>> Fact checkers?

0:46

And so with that

0:46

>> teacher who will fact check that one.

0:48

>> I actually thought TJ was going to fact

0:49

check me, but he fact checked me

0:51

incorrect.

0:52

>> It didn't seem 10, but that it could be,

0:53

could be.

0:54

>> Bunch of you know who's read Revelation.

0:56

What the hell's going on here, guys?

0:57

>> Hold on, hold on. Well, that is kind of

0:59

what they what it answers, you know.

1:01

>> [laughter]

1:02

>> Anyway.

1:03

>> All right, so he has 10 horns. Called

1:04

it. Okay, primary reference, I got it.

1:06

Okay, I was I was correct on that one.

1:08

>> Okay, good.

1:09

>> Referenced. All right, so with that in

1:11

mind, we

1:13

>> for our theology. All right.

1:14

>> [laughter]

1:15

>> Just let it happen, everybody, okay?

1:19

My gosh. All right, this is not starting

1:21

off the way I wanted it to. So anyways,

1:23

nor does any standup. All right, so with

1:24

all of that said, that means as part of

1:27

our amazing content, I have now

1:29

organized us an amazing linear board,

1:32

which will help us walk through a more

1:34

organized standup, because if there's

1:36

one thing I know about in raising a

1:39

series A is they want to see

1:41

agile, okay? I'm pretty sure that's

1:43

correct still in today's day and age, so

1:45

we're getting really agilized. No, it's

1:47

actually going to be it's going to make

1:48

us hopefully have a more organized and

1:50

amazing standup, so that the world's

1:52

most attended standup, which is our

1:53

current standup right now, right now,

1:55

just in case you're wondering.

1:57

You know, you may not know this, Casey.

1:59

There's There is

2:00

There's 2,500 people on the stand-up

2:03

right now. This is the world's most

2:04

attended stand-up.

2:05

>> Oh.

2:05

>> It's like you couldn't even like it we

2:07

previously no one knew if that many

2:09

people could even stand at the same

2:10

time.

2:11

>> Yes. Yes. Yes. Yes. Yes. Yes. Yes. Yes.

2:12

Yes. Yes. Yes. Yes. Yes. Yes. Yes. Yes.

2:12

Yes. Yes.

2:13

>> We shattered records. Okay.

2:14

>> Oh, yeah, definitely. What was the

2:15

record before? Like 100 people, maybe?

2:16

>> Yeah, like 100 to 200, I don't know.

2:19

Maximum.

2:19

>> Now we're at 2,500 is with world It's

2:21

It's world shattering.

2:22

>> Yes, totally. It's a game changer.

2:25

>> Yeah. I want to add, though, we didn't

2:26

get an invite or the topics until this

2:27

morning, like 2 hours

2:28

>> No, yeah, so this

2:29

This stand-up's going to be amazing cuz

2:31

cuz Prime's like, uh

2:33

we're going to talk about this new Well,

2:35

you know what? I won't spoil it, but

2:36

let's put it put it this way. It

2:37

probably was a topic that would have

2:39

been nice to research beforehand if

2:41

we're going to talk about it. That's

2:42

That's all I'm going to say. I don't

2:44

want to spoil anything.

2:44

>> I had to listen to Jim listen to YouTube

2:46

videos about it. Would have BEEN NICE.

2:47

>> [laughter]

2:48

>> WELL, YOU KNOW, THAT

2:49

>> That makes this authentic, uh Casey. You

2:51

get a react to Trash's workout gym

2:54

knowledge.

2:54

>> Okay.

2:55

>> Right. Rather than that, Trash is the

2:56

expert. You get Trash back.

2:58

>> I am not the expert.

2:59

>> Trash is the expert. Trash is the

3:00

expert.

3:01

>> I was like, "Oh, no."

3:02

>> Dude, if there was a trash fact every

3:04

episode, that would be That would be

3:06

fire.

3:06

>> It's It's literally going It's going on

3:08

the linear board.

3:09

>> I literally have endless lore since

3:11

then.

3:11

>> Um another thing that I would like to to

3:14

uh ask at this point is if we are

3:17

serious about this series A, which I

3:20

think we should be,

3:21

uh is it time for us to start putting

3:23

together a pitch deck?

3:25

>> Oh, wow.

3:26

>> don't know what one of those are, but I

3:28

think we need to do that. I think you're

3:29

I think you're going to do it.

3:31

>> We need some slides.

3:32

>> My startup, dailybjj.ai,

3:34

is going nuclear.

3:35

>> Congratulations on product market fit.

3:38

>> Sorry I called you the Diffeler.

3:40

>> I knew you weren't the Diffeler. I

3:42

always thought it was the intern with

3:43

the weird glasses.

3:49

>> You should have seen his code.

3:50

>> But anyways, if you want to get in the

3:52

ground floor of this thing, we're taking

3:54

angel investments.

4:12

>> [laughter]

4:16

>> So, you finally cracked the case, Merge

4:19

Cop.

4:19

>> You put the white space in the diff. You

4:22

made the synthetic traffic.

4:24

You made me approve that PR. You're the

4:26

diffler. I ALWAYS KNEW IT.

4:30

>> [screaming]

4:31

[laughter]

4:33

>> YOU'VE MERGED YOUR LAST PR, DIFFLER.

4:37

>> I already clicked merge on my PR 35

4:39

seconds ago.

4:40

>> Huh?

4:42

>> My merge is blocked.

4:44

CURSE YOU, CODE RABBIT.

4:47

>> YOU SAVED THE CITY from a code

4:48

review-related crime.

4:50

>> Thanks, Commish.

4:51

>> Well, technically, it was Code Rabbit

4:54

that saved the day. Not only do they

4:55

have advanced AI features that can

4:57

detect security vulnerabilities, like

5:00

what the diffler was trying to merge,

5:01

they also have ways to enforce styling,

5:03

[music] linting, and a variety of other

5:06

tools. And so, you can stop wasting time

5:09

reviewing code that humans don't need to

5:11

review.

5:11

>> You can try it yourself at

5:13

coderabbit.ai.

5:15

>> So, Merge Cop, I would say it's a little

5:18

misleading to say you did it all by

5:20

yourself.

5:21

>> Huh. Would you like some cake?

5:23

>> Oh, sure. Thanks.

5:28

>> All right. So, Casey, I saw this tweet

5:30

and I figured you could maybe give me

5:31

like a little I think I understand the

5:34

purpose here

5:35

and what the problem is of it. But, I

5:37

thought maybe you could help me a little

5:39

bit on this one. This is my blocker, by

5:40

the way, for the day. Which is I saw

5:42

this where someone said, "Hey, don't use

5:45

division. Use reciprocal math instead.

5:48

>> Okay.

5:49

>> Can you can you can you can you maybe

5:51

give us like us dumb [clears throat]

5:52

people why this is either good or bad

5:54

idea?

5:56

>> Oh boy.

5:57

I have no idea what that means.

6:00

>> Well, right here he's doing array

6:03

index I divided by pi versus array index

6:07

I multiplied by the reciprocal. So, 1

6:09

divided by pi. So, he does the division

6:12

thing once and then hits it with a bunch

6:14

of multiplications.

6:16

>> Yeah. Um

6:18

So, there's a lot of things that you

6:20

would want to talk about here and that's

6:22

I actually saw that tweet that you're

6:24

talking about. Yes. And unfortunately

6:26

>> your account?

6:28

>> [laughter]

6:28

>> No. I mean, see here's the here's the

6:30

problem, right? So, delete your account.

6:32

I love delete your account and I love

6:34

what Ryan's doing there.

6:36

>> [laughter]

6:36

>> But what mostly what he's doing there is

6:39

he's just like

6:41

and he even described this on one of his

6:42

own streams. He's just saying like look,

6:45

basically like you guys are all the

6:47

people who like come up and and and beg

6:49

for change on the street or something.

6:51

Like you're just totally making the

6:52

world difficult to live in by like

6:54

trying to get all of like just just

6:57

hounding everyone who goes into a

6:59

particular area to try and get them to

7:01

give you something, right? Which is your

7:03

attention or whatever. They're they're

7:04

like these AI shill accounts or

7:05

whatever. People trying to get

7:07

engagement. They're trying to get

7:08

follows, whatever it is, right? And it

7:10

just really degrades the experience of

7:12

being in that place. That's basically

7:14

what he said on like one of his streams.

7:16

I'm probably phrasing it poorly. Um I I

7:19

think he

7:20

>> [laughter]

7:21

>> he said I think he said the phrase

7:23

"If you go to a third-world country like

7:26

France." Was the WAS [laughter]

7:28

LITERALLY I THINK the phrase he used.

7:30

>> I love Ryan.

7:31

>> in that stream. So, I don't know don't

7:33

don't blame me. Don't come after me for

7:36

this. I'm just repeating what I believe

7:38

it was.

7:39

Um

7:40

>> [laughter]

7:40

>> But anyway, the point is just that uh

7:42

that was sort of the point of delete

7:44

your account. And I And I'm He's right.

7:46

Like, there's just all of this kind of,

7:47

you know, really bad behavior that's

7:49

going on there.

7:50

And the uh this tweet I don't really

7:54

like being harsh on people who are

7:56

trying to talk about performance because

7:58

I'm glad that we somehow went, you know,

8:00

uh over the past, you know, 10 years or

8:02

so to people actually starting to think

8:04

that it matters that you're thinking

8:06

about how the CPU is going to do things.

8:07

So, I don't want to like be mean to

8:09

anyone who's at least trying to say

8:12

like, "Hey, let's think about what this

8:14

is doing." The problem is that it

8:16

requires some actual knowledge to like

8:19

actually get these things right. And a

8:21

lot of times, if you just ask the AI or

8:23

if you just read something on a blog and

8:25

you then

8:26

repeat that, it's not accurate, right?

8:29

Because performance is a very specific

8:30

thing. It's It's kind of like repairing

8:32

a car or something. You can't just know

8:34

like one thing about like, "Oh, yeah,

8:36

you know, the carburetor or something."

8:38

Right? It's like, you have to actually

8:39

know how the whole thing works.

8:42

Right? [laughter]

8:43

>> Carburetor.

8:44

>> car is short for. No, me exactly. Like,

8:45

I'd use it I'm using that as as an

8:47

example. Like Like, I'd just be like,

8:49

"Uh the spark plug." Or Right? You know,

8:50

it's like, I don't even really know much

8:53

beyond like there's a thing that ignites

8:54

the gas, right? Then okay, that's it. I

8:56

couldn't tell you like is it broken? Is

8:58

it working properly? Does it need to be

8:59

tuned? What's going on? Right?

9:00

>> Spark plugs are for electric cars,

9:02

right? Cuz you get it from the battery.

9:03

>> I mean, that's There you go. That's Must

9:06

be. I mean, clearly.

9:08

So, yeah, this particular tweet is not

9:10

is not good though because it misleads

9:13

people, I think, on a lot of points. I'm

9:15

not sure if the person actually tested

9:17

this. They've got like seconds listed

9:19

there, which is a little weird cuz I

9:22

don't The Those numbers um

9:25

I I mean, there's there's a lot of

9:27

things that could have happened there. I

9:29

don't know. So, I don't know the

9:29

circumstances of it. But, I just don't

9:31

want people to be misled.

9:34

When you're looking at a loop like this

9:36

both of those loops that are in that

9:37

tweet, cuz I remember this. Of course, I

9:39

can't read what's on prime screen cuz

9:41

Riverside won't let me, but I remember

9:43

it fairly well.

9:45

You're basically going You're going

9:46

through arrays

9:48

and it's basically a read-modify-write,

9:50

right? It's It's loading a value out of

9:52

array. The value is a floating-point

9:53

number.

9:54

Um loading a floating-point value out of

9:57

an array.

9:58

It's modifying that floating-point

10:00

number and writing it back to the array,

10:02

right?

10:03

And uh the operation that's being done

10:06

on the array, the only difference

10:08

between the loops is one of them is

10:11

doing a divide, so it's loading,

10:13

dividing, and rewriting. And the other

10:15

one is loading, multiplying, and

10:17

rewriting.

10:18

>> Correct.

10:19

>> Now, obviously, in math

10:22

if you do a uh divide of a number, if

10:26

you say, "I'm going to divide by X."

10:29

There's no difference between that and

10:30

saying I'm going to multiply by one over

10:33

X,

10:34

right?

10:35

>> True.

10:36

>> True.

10:36

>> That's true in math, meaning in like in

10:38

grade school, you write it down. If

10:40

someone said, "Oh, you know, A divided

10:43

by

10:44

What's sorry?

10:45

>> Okay. All right. All right. [laughter]

10:47

TJ was a math major, so okay, he's

10:50

flexing that he knows these things,

10:51

okay.

10:51

>> Math major, get out of here.

10:53

>> Yeah, TJ.

10:55

>> This is This is like in in like junior

10:57

high, right? You Or earlier probably,

10:59

right?

10:59

>> Yeah, yeah, totally junior high, yep.

11:01

>> Yeah, totally.

11:02

>> [laughter]

11:03

>> Come on, guys. LIKE SERIOUSLY, LIKE I AM

11:06

NOT GOOD AT MATH. I AM NOT GOOD AT MATH.

11:08

IF I know this, you know this. Come on.

11:10

>> Yes, Casey, this is true. This is true

11:12

and and straightforward, yes.

11:14

>> Um this is just something people did in

11:16

grade school, right? And so uh the idea,

11:19

this was certainly true in the old days

11:21

um for computing, right? Is Look, if I

11:24

have floating point, right? If I have

11:26

the ability to use floating-point

11:27

numbers. Integers, obviously, if we're

11:29

using integers, then you have to do a

11:31

lot of much more crazy stuff if you

11:33

wanted to divide instead I mean multiply

11:35

instead of divide. Go into that cuz

11:38

that's a whole 'nother ball game. But if

11:40

assuming you're in floating point, then

11:41

that means that you could literally just

11:43

store the number one over X, whatever

11:46

you're going to divide by, and then

11:48

multiply by it instead. So if you're

11:50

going to do a bunch of multiplies, this

11:52

is potentially an attractive option

11:54

if dividing is slow, right? So if we

11:56

look at divide and we go the dividing's

11:58

too slow, we could just do it once and

12:00

then multiply if the if the multiply is

12:02

faster. Now, it's worth noting that

12:05

these are not necessarily the same

12:06

operation because in floating point,

12:09

right? An actual divide, this divided by

12:12

that, right?

12:14

May produce a different answer because

12:16

you don't know like when you do it an

12:18

inversion like that, you're only

12:19

capturing part of the answer. Like the

12:22

answer might have gone on for quite some

12:23

time and you're only getting a smaller

12:25

part of it. So you're not doing infinite

12:27

precision math and so you also have to

12:29

be aware of the fact that this will

12:31

change the answer. So you have to you

12:33

have to know, right? If that mattered to

12:35

you or not when you're doing this. So if

12:38

it's just some random thing you're doing

12:40

for some UI element or something like

12:42

that, then it probably will never

12:43

matter. If this is scientific computing,

12:46

it could be a very big difference. And

12:48

so this is the first thing that I want

12:50

to point out is that this is not a thing

12:52

you can just tell somebody always invert

12:54

your numbers and then multiply because

12:56

that may be very bad advice in certain

12:58

circumstances. That's thing one that

13:00

wasn't mentioned in the tweet that you

13:01

have to understand.

13:03

>> The butterfly effect.

13:05

>> Uh yeah, well, it's yeah, basically

13:06

like, you know,

13:07

>> Jeff Goldblum with the water going down

13:09

his hand effect.

13:10

>> It might even just be the

13:12

>> Good movie.

13:12

>> the the the 1,000 mile butterfly. Like

13:16

it's this huge butterfly because it

13:17

could be literally it could just be just

13:19

literally the answer you get right here

13:21

could be wrong enough that you care,

13:23

right? Depending on the circumstances

13:25

and depending on what those input

13:26

numbers were and what their scales were

13:27

and all this stuff, right?

13:28

>> Casey, can you just I there is a decent

13:31

chance a few people in the audience

13:34

don't understand why that happens on a

13:36

computer.

13:38

>> Well, so to be fair, I'm the wrong one

13:41

to explain it because I don't work on

13:43

scientific computing, right? And people

13:46

who work on scientific computing are

13:47

much better at analyzing things like,

13:50

you know, what's called ULP or basically

13:53

like the last what happens to the final

13:55

bits

13:56

in floating point numbers and things

13:57

like that. But in general, when you're

13:59

working with floating point on a

14:00

computer, what's actually happening

14:02

there is that you're taking the size of

14:06

the storage that you, you know, made for

14:08

those numbers. So let's say that you're

14:10

doing 32-bit floating point numbers. In

14:12

scientific computing it might be 64-bit

14:14

numbers, whatever it is.

14:16

If you just have 32 bits, you only have

14:18

a certain amount of space to store what

14:21

you would prefer was an infinite series.

14:24

Like let's say we want to compute with

14:26

pi. Right? Everyone knows from grade

14:28

school pi is this infinitely long

14:30

series. Like there is no storage that

14:32

can store pi. It just keeps going.

14:35

So we have to approximate that, right?

14:38

By putting it into some format. And all

14:40

the numbers that we work with in

14:42

floating point computing, they have this

14:43

kind of format. Just something simple.

14:45

>> you for a quick second? Trash, when he

14:47

says pi, he's not talking about a snack.

14:49

Just in case you're wondering.

14:51

>> I don't even like pi, actually. I

14:52

actually hate [laughter] pi.

14:53

>> Me neither. Trash, yes.

14:55

>> There's another reason Trash is the star

14:56

of the stand-up. No pi zone.

14:58

>> Yeah, that's true.

14:59

>> Yeah, what's your favorite pi adjacent

15:02

kind of thing? Like is it cake or

15:03

brownies or like what is it then?

15:04

>> Love brownies.

15:05

>> Pi is not adjacent to cake and you'll

15:07

take that back. We can We know we just

15:09

got to move on cuz that was very

15:10

offensive.

15:10

>> Okay, that's it. Put that next next

15:12

episode.

15:13

>> I'm putting it in the backlog. How

15:14

adjacent How adjacent [laughter]

15:17

is cake to pi?

15:20

>> Okay, so

15:21

>> anyway, if you if you look at what

15:22

happens to these things during all your

15:24

computations, every computation that you

15:26

do is going to have some infinitely

15:28

precise answer that's the answer you

15:29

actually wanted and that you will want

15:31

you would have wanted to carry through

15:33

all of your subsequent operations,

15:35

but at every step of the way, the

15:37

computer has to like round it basically

15:40

to whatever will fit back into that

15:43

32-bit encoding. And that 32-bit

15:45

encoding is basically, you know, it's a

15:47

sign bit that says whether the thing's

15:49

positive or negative. It's some number

15:51

of bits for the exponent, right? Which

15:54

says

15:55

>> I think is

15:56

the sign.

15:57

>> is the next thing.

15:58

>> Oh, the next thing. That's right. That's

15:59

right.

16:00

>> Uh the exponent basically says what the

16:01

scale is, like how big roughly is this

16:03

number.

16:04

And then the mantissa are the bits that

16:07

actually say like what the number is.

16:08

So, the reason it's called floating

16:11

point is because it's broken up into

16:13

these pieces.

16:14

Floating point, the point is the decimal

16:17

point, if you will. The floating part is

16:20

that we store an exponent, so we say

16:23

where the decimal point is, and then the

16:25

mantissa is just the actual digits. So,

16:28

we're basically floating the point to

16:30

where it needs to go using the exponent,

16:31

right? And the reason we do this is

16:33

because we're trying to compute with

16:35

extremely large and extremely small and

16:38

everything in between, and so we can't

16:39

just take 32 bits, which has a very

16:42

small range if we just use an integer,

16:44

right? If we fixed point, if we just

16:45

said the decimal point is here, it would

16:47

kind of suck. That's what floating

16:49

point's for. It's why we use it for

16:50

everything that isn't just nice, you

16:53

know, typical whole number kinds of

16:54

stuff that we're doing, right? You know,

16:56

usually when we when we use

16:58

uh just answer things like that, just

17:00

integer numbers.

17:02

>> The common The common one that people

17:03

laugh at, right, is it's what 0.2 + 0.1

17:07

in JavaScript, and you get the 0.299999,

17:10

right? So, it's like this is This is why

17:12

that happens, though. So, just I feel

17:14

like there's probably a decent amount of

17:15

people who are like just haha JavaScript

17:17

stupid. I mean, you're right, but like

17:19

that's not that's like a right for the

17:21

wrong reason JavaScript problem.

17:23

Exactly.

17:24

>> That works in C, it works in Java, it

17:26

works in all of them.

17:27

>> Yeah.

17:29

>> yeah, JavaScript is weird in that it

17:30

kind of only has double precision

17:32

arithmetic period. Like it's just like

17:34

that's just what we do everywhere, I

17:36

think.

17:36

>> Yeah, unless you use bitwise operations,

17:38

then it converts to a signed 32-bit

17:40

number.

17:41

>> No, it will just magically at that time

17:43

>> one moment you can actually do a you can

17:44

actually do a a overflow

17:46

at 32 bits.

17:48

Only if you use uh bitwise stuff.

17:50

>> Again, this is always why JavaScript

17:51

compilers are so hard, right? Like cuz

17:53

they have to go like, "Oh, I don't you

17:55

don't want to do double precision

17:56

arithmetic for like your loops and like

17:58

counter and things like that." So, it

17:59

has to analyze and go, "Okay, I can turn

18:02

this into an integer for this code."

18:04

Right? It's got to do that work. Normal

18:06

languages, it's already knows, right?

18:08

It's just been told like this integer

18:10

doesn't have to think about that, right?

18:11

But anyway, so

18:13

that's that's the floating-point math

18:15

thing. And that's sort of what this this

18:17

uh this snippet was trying to say is

18:19

just invert the floating-point number so

18:20

you only pay for the divide once, then

18:22

multiply by it cuz that'll give you the

18:24

same answer. Again, the tweet didn't say

18:26

no, it's not quite the same answer, and

18:27

it might matter depending on what you're

18:29

doing and depending on what the inputs

18:30

are.

18:31

But anyway, uh the other problem I have

18:34

with this is it misrepresents the

18:35

performance as well for a number of

18:37

reasons.

18:39

So, the first reason it mis-

18:41

misrepresents the performance is because

18:43

it has some weird statements about how

18:44

long a division is. So, it said

18:46

something like 20 to 40 cycles for a

18:49

divide or something like that,

18:51

I think.

18:52

>> Yes, 20 to 40.

18:54

>> And I just have no idea where that's

18:55

coming from, right? Like I've no idea

18:57

where they got that number from. They

18:58

like I really wish it had said where it

19:00

got that number from, right?

19:02

>> came from his heart, all right?

19:03

>> It came from his

19:04

>> Yeah.

19:04

>> Yeah.

19:05

>> He could feel it.

19:06

>> He could feel those cycles.

19:08

>> The CPU whisperer.

19:09

>> Yeah.

19:10

>> So, a floating-point divide

19:13

um

19:14

you know, if you look So, if you look at

19:16

a modern chip, like let's say a Zen a

19:18

Zen 4 or Zen 5 chip, right? Something

19:21

that you might be running today that

19:22

might be in a computer you'd buy today.

19:25

Typically, their floating point divide

19:26

units are extraordinary.

19:30

Very fast. I want to say that they can

19:32

complete these things in I don't know,

19:34

five, six cycles, something like five

19:36

cycles, maybe. I'm not sure what it is.

19:38

We should probably look it up. But, 40

19:41

is no no where to be found. I mean,

19:44

that's it's not quite off by 10, but

19:46

it's something like 10. Like, it's very

19:49

very wrong.

19:50

>> I think it's from uh ChatGPT cuz if I'm

19:52

not mistaken, I actually asked this

19:53

question not too long ago and I got the

19:55

answer of like eight to 30 or something

19:58

like that cycles for a divide depending

19:59

on the CPU and architecture.

20:02

>> I mean, the part the part that the

20:04

ChatGPT got right is the depending on

20:06

the the architecture part cuz it

20:08

[laughter] does depend.

20:08

>> Okay, okay.

20:10

>> Uh but like let's sit here, I'll just

20:11

look it up on my machine right now.

20:12

Like, we'll just see what roughly it is.

20:14

So, if I go in and I say I want to do a,

20:17

you know, 32-bit um

20:19

uh like we're going to do a div PS here

20:21

or a div uh SS, let's say. And we look

20:24

at what's going to happen on a Zen 5

20:25

chip, um the the total latency, right,

20:29

is less than 10 cycles for for that

20:32

operation, latency, right?

20:35

So, that means that if you

20:37

>> about multiply then?

20:38

>> Sorry?

20:39

>> What what about for multiply?

20:41

>> Multiply will be four.

20:43

>> Okay.

20:44

>> I want to say, right?

20:46

Um so, three three on the Zen 5.

20:48

>> though, Casey? We just need to know how

20:50

many [laughter] FPS's 10 cycles.

20:52

>> Right.

20:52

>> If it were a bouncing ball

20:54

>> Casey, how fast is that ball?

20:55

>> [laughter]

20:57

>> So, three on that and I think four four

20:58

is on my CPU, which is Zen 4. I think it

21:00

should be four, right?

21:02

Uh is that correct? Zen 4. Also three.

21:05

So, all right, this is better than I

21:06

would have expected. I'm used to four,

21:08

these get three,

21:10

right?

21:11

Now, so in general,

21:14

that is the number that you would use if

21:17

you were trying to analyze how long it

21:20

takes you to get the answer back, right?

21:24

These loops don't care how long it takes

21:27

to get the answer back because nothing

21:28

is waiting for that, right? On the

21:31

iteration of the loop, as you go

21:33

through, right?

21:36

You're going to do a load,

21:38

the floating point op, either divide or

21:41

multiply, and then the store,

21:44

right?

21:45

When you do that, the next iteration of

21:48

the loop doesn't have to wait for that

21:51

to complete. So, those are just in the

21:53

scheduling queue going, they can be

21:56

dispatched. The next iteration of the

21:58

loop is already decoded and probably

22:01

also in the scheduler, given this is a

22:03

hot loop, presumably we're just running

22:05

about, you know,

22:05

>> It's good looking at that. It's good

22:07

looking, for sure.

22:08

>> It's [laughter] just cooking.

22:09

So, these are all going to pile up in

22:11

the scheduler, scheduler's going to

22:12

dispatch them every cycle as it can.

22:15

And so, what we care about is not how

22:17

long it takes to get the answer cuz no

22:18

one cares.

22:20

What we care about is how long will it

22:22

take us to issue the next one. So, if we

22:25

issue them on this cycle, how long does

22:26

it take us to issue the next one, right?

22:29

And so, that's what we typically call

22:30

that's a throughput number. So, we

22:32

typically call that, right?

22:33

And if you look at the throughput

22:35

numbers for multiply, right?

22:39

Typically, you can issue two floating

22:41

point multiplies per cycle,

22:43

which means it takes half a cycle

22:45

effectively to multiply. Half a cycle,

22:48

okay?

22:50

That is what you That is the number you

22:52

should be listing or thinking about when

22:54

you're looking at this loop.

22:55

For divides, the throughput again is

22:59

remarkably good on modern CPUs, so it's

23:02

like in the two to three range.

23:06

So, is multiply faster than divide?

23:10

Sure, right? Half a cycle versus three

23:13

cycles when we're analyzing this loop,

23:15

let's say, or something along those

23:16

lines.

23:18

But then you have to ask the question,

23:20

do you care?

23:21

This is a load

23:23

a op

23:25

and a store.

23:27

Unless you're entirely out of the L1

23:29

cache, which is going to service, you

23:31

know, at about, you know, like let's

23:33

say,

23:34

uh one, you know,

23:36

it's probably like two per cycle, three

23:38

per cycle, it could get you answers back

23:40

from the L1 cache, right?

23:42

If you're talking about the L2 cache,

23:45

then, I mean,

23:47

I don't know, the number of clocks it's

23:49

probably going to take 14 cycle latency,

23:51

something like this.

23:53

If we're looking at how long it takes

23:54

you to actually get the memory back,

23:57

it's not clear to me that unless the

23:59

unless the CPU was predicting extremely

24:01

well what you were going to do and

24:03

always had things, like you'd have to go

24:05

look at the actual bandwidth to see if

24:06

you were actually going to get enough

24:08

back such that you could operate them on

24:10

them at that speed. Maybe you can.

24:13

It's it's not in my brain right now as

24:15

to whether you would actually be able to

24:16

do that at a speed that would matter at

24:19

the like, let's say, three cycles, uh

24:22

throughput number that you're going to

24:23

get on the divide.

24:25

So, it really just it was very

24:27

misleading, the whole thing, and

24:30

as I you know, you heard me say all that

24:31

stuff,

24:33

I would want to go look at this loop and

24:36

and check it first, and I'd want to

24:38

think about it a little bit before I

24:39

would post anything like that, and it

24:40

was just that clearly wasn't done.

24:42

Because the none of those numbers make

24:44

sense that are placed there. Is it true

24:46

that multiply is faster than divide?

24:48

Yes.

24:49

Does it even matter in this case? No. I

24:51

don't think it probably does, but maybe.

24:54

And furthermore, usually, in most cases,

24:57

right?

24:59

When you're looking at actual loops,

25:00

more is going on.

25:02

And so, it's also going to mislead

25:04

people. So, point number three, it's

25:06

also going to mislead people in this

25:08

final way, which is to think that they

25:10

always have to do this. It's the This is

25:12

the problem with these kind of like out

25:13

of context performance things. People

25:15

read it, they think, "Oh, I just need to

25:17

invert all my numbers, and then my code

25:19

goes fast." And then they go around

25:21

inverting all of their numbers and

25:22

turning them into multiplies. And the

25:24

problem with that is in a lot unless you

25:27

were unless you had serial dependency

25:29

chains that really could have been

25:31

unblocked by this change,

25:33

you're just wasting your time. The

25:34

divide would have been just as fast. It

25:36

would have overlapped with other things

25:37

you're doing in the loop. You were

25:39

The thing was waiting on a on an

25:41

uncashed read or something, so the whole

25:43

loop was was waiting 80 cycles anyway,

25:45

so none of this mattered, right? So, the

25:48

problem again is that like performance

25:50

is a thing, performance is a process.

25:53

It's an understanding. You have to

25:54

understand all those things that I just

25:56

said. They're not actually that hard to

25:58

understand. You just You can't You just

26:00

need to learn them. You need to spend a

26:01

few weeks learning this stuff. And then

26:03

you can think about do I need to invert

26:05

this number? Do I not need to invert it

26:06

at that point. What you don't want to do

26:08

is try to package it into a tweet like

26:09

that when you haven't really done the

26:12

work of understanding why it matters or

26:13

anything like that because other people

26:15

will take away the wrong lessons. So,

26:17

that was a very long-winded way of

26:18

trying to explain everything I didn't

26:19

like about that tweet. Again, no

26:21

offense. Sorry to the person who posted

26:23

it. It's just I don't think that kind of

26:24

thing is helpful. It's sort of like

26:25

saying always call memset or never call

26:27

memset or these other weird performance

26:29

things that people pass around, which

26:31

out of context don't help anybody

26:33

because you're just going to make

26:35

incorrect decisions if you think that

26:36

the key to performance is always do X or

26:39

never do X. That's not how it works.

26:42

Spark plugs.

26:44

>> Great job, Casey. Thank you.

26:47

>> I did my best.

26:48

>> I figured you'd be able to help me on

26:49

that one because I had the rough idea

26:51

that it would introduce error, but I had

26:52

no idea to the extent that divide and

26:55

multiply may not make any sort of actual

26:56

real difference in your program.

26:59

Plus all the crappy things I do anyways

27:01

probably destroy my program's

27:02

performance long [laughter] before we

27:03

get to this loop.

27:04

>> Yeah, Prime, we've got other issues

27:06

before we get to whether we're

27:07

multiplying or dividing, I think. Yeah.

27:10

>> I just really want to take my Lua

27:11

program seriously, TJ, okay?

27:13

>> [laughter]

27:13

>> It's It's usually I'll The one thing I

27:16

can put in someone's head that's

27:17

generally not wrong is

27:19

Don't confuse integer multiply with

27:22

floating-point multiply. Floating-point

27:24

multiplies are much faster than integer

27:26

multiplies typically. And so some of the

27:29

time when you see people saying, "Oh my

27:30

god, in like divide is so bad."

27:33

Sometimes they're thinking about integer

27:34

divide. And there's a lot of reasons

27:36

There are a lot of reasons why integer

27:39

divide is bad. A lot of reasons.

27:41

>> to put that on the block list for next

27:43

week. You can We can We can talk about

27:44

that cuz that's actually a

27:46

That seems very interesting.

27:47

>> Hey guys, if you like this episode, you

27:49

can watch the rest of it on Spotify.

27:51

And don't forget to like AND SUBSCRIBE.

27:53

WOO!

27:55

SEE YOU LATER.

27:56

>> [singing]

27:58

>> Five code errors on my screen. [music]

28:03

Terminal coffee

28:07

Living the

28:09

dream.

Interactive Summary

This video features a lighthearted team discussion regarding startup culture, project management, and a technical debate over a performance-optimization tweet. The team discusses the potential pitfalls of blindly applying performance advice—specifically the suggestion to replace floating-point division with multiplication by a reciprocal—while also touching upon how modern CPU architecture handles these operations.

Suggested questions

2 ready-made prompts