HomeVideos

Linus Lays down the Law

Now Playing

Linus Lays down the Law

Transcript

382 segments

0:00

Hey there, buddy. Do you know why I

0:01

pulled you over? Undisclosed use of AI

0:04

while coding. Yeah, that's an actionable

0:06

offense you need to site when you use an

0:09

LLM. Okay, Buster. If you continue this,

0:11

we're going to throw you clean out of

0:13

the city. Okay. Yes. This is my mason

0:15

jar creeping into the the camera angle.

0:19

Is it a hot cup of pee? Yes, it is. Now,

0:21

you're probably thinking, "Okay, what

0:22

the heck are you even talking about? Why

0:24

are we talking about needing to publicly

0:27

uh announce that you're using an AI?

0:29

Well, the reason why we're talking about

0:31

it is that the Linux development team

0:34

and Lionus himself has finally come to a

0:36

conclusion of how LLMs can be used

0:38

within the Linux kernel development. And

0:40

for the last 6 months, there has been a

0:42

bit of a raging war. Okay, there's a

0:44

France Fertan involved that kicked off

0:46

the entire thing. And there's lots of

0:48

angry and sweaty nerds in the comments.

0:51

So to kick this thing off, I think it

0:52

would be best if we just looked over the

0:54

code in which kicked off this entire

0:55

thing. It's only a little 19line change

0:58

in which there's a replacement of this

1:00

old hash table that used to be used, but

1:02

instead there's now this new define

1:04

hasht function. Instead of saying how

1:06

big it is numerically, you say it as a

1:08

power of two. 2 to the 7 128. So you

1:10

say, "Hey, I only need seven bits."

1:12

Instead of creating a key by doing a

1:14

little bit of masking operations,

1:16

instead you just simply get to use some

1:17

of the more convenient functions. Same

1:19

thing with adding. Deleting is virtually

1:22

the exact same. So, even me as a layman

1:24

looking at this code, this looks rather

1:26

reasonable, right? But if you zoom in,

1:28

no, we're talking about zooming in.

1:30

Okay. No, no, no, no, no, no. You're not

1:32

listening to me. I want you to get in

1:33

it, baby. Do you see that read mostly?

1:35

Do you see how that read mostly was

1:37

removed? Well, a bunch of the reviewers

1:38

didn't really catch that. I assume they

1:40

thought that the define hasht uses read

1:43

mostly. Oh, but it didn't. And this

1:45

caused a performance regression. Now,

1:47

performance regressions. Oopsy daisies.

1:49

Changing of interfaces, usually not a

1:51

big deal. This is just kind of part and

1:53

parcel for developing any large

1:55

sophisticated piece of software. But

1:57

there were several people that got quite

2:00

hot and bothered. We're talking about

2:01

frothing here, okay? They were out there

2:04

just yelling, pulling a Jerry Maguire,

2:06

just throwing his briefcase everywhere.

2:08

And the thing is is that the performance

2:10

regression and the patch actually really

2:12

didn't cause any problems. Yeah, people

2:14

had to fix some stuff, but no one was

2:16

really upset. What actually caused the

2:18

whole thing was this article right here

2:20

on June 26, 2025, which is talking about

2:24

supporting kernel development with large

2:26

language models. And this was actually

2:28

just recapping a talk in 2025 from the

2:30

open-source summit given by Sasha Leven.

2:33

In this talk, he kind of talks about how

2:35

he's utilizing LLMs for kernel

2:37

development. In one of the examples, he

2:38

points to that patch I just got done

2:40

showing you that was merged in 6.15. But

2:43

here's the thing is that when the

2:45

Colonel developers inevitably found this

2:47

just a short bit later, they said, "Hey,

2:49

wait a second. That patch he's pointing

2:51

to, I knew about that patch. I reviewed

2:54

that patch. I am baffled that he

2:56

apparently saw no need to disclose this

2:59

when he was posting the patch. Or am I

3:01

missing something in the lore links?"

3:03

And this, my friends, is how World War

3:05

AI happened. Now, there's obviously a

3:08

lot of conversation going on, going back

3:10

and forth, disputing the legal

3:12

challenges of using AI, which honestly,

3:16

I'm on their team. There is definitely

3:17

some gray area on the old copyright,

3:19

clean room engineering. Can you ever

3:21

have clean room engineering if the AI

3:24

itself likely has the copyrighted code

3:27

actually stored within its weights or

3:29

compressed within its weights? Yeah,

3:30

that's a that's a serious problem. And

3:32

that is a serious legal problem to the

3:34

point that even Microsoft Copilot comes

3:37

with the customer copyright commitment.

3:39

It's a provision in the Microsoft

3:41

product terms that describes Microsoft

3:43

obligations to defend customers against

3:45

certain third-party intellectual

3:46

property claims relating to output

3:48

content. Microsoft is stating and

3:50

willing to go to bat for its customers

3:52

using C-pilot if C-Pilot leads to

3:55

generated copyrightable and enforceable

3:58

code. But that's only part of the reason

3:59

why a lot of people were upset. A big

4:01

part of it is that if I would have known

4:04

that it was AI generated, I would have

4:06

reviewed it more strongly. I was going

4:09

to even ask Sasha if this came from some

4:10

new tool. I think I should have. And

4:12

yes, it would have been nice if Sasha

4:14

mentioned it was completely generated by

4:16

an LLM because I would have taken a

4:17

deeper look at it. It appears from the

4:19

comments below that it does indeed have

4:22

a slight bug which I WOULD HAVE CAUGHT

4:24

if I would have known this was 100%

4:26

degenerated. And this honestly to me

4:28

this is kind of like an interesting

4:29

point. I I you know what I will have to

4:31

first and foremost say that the people

4:33

developing Linux are obviously extremely

4:36

talented engineers and I think it's

4:38

completely reasonable to say that

4:39

they're likely better engineers than I

4:41

am. But this idea that you would have

4:44

reviewed it different knowing that it

4:45

was an LLM versus someone you know. I

4:48

don't know about that. Why why would you

4:49

ever feel that way? Isn't a Isn't that

4:51

the purpose of a review for you to put

4:53

on your thinking cap and to think deeply

4:55

about the code being merged such that

4:57

you make sure that it is of the quality

4:59

you would expect? And so if somebody

5:00

hands me 19 lines and it's my job to say

5:04

yes, this code is good or no, this code

5:06

is bad, I really don't care who hands it

5:09

to me. I'm going to do a review and

5:11

ultimately even me and my layman's I

5:13

would have been able to see that, hey,

5:14

these two are different. I would have

5:16

had the check. Does define hasht come

5:19

with this intrinsic flag read mostly? I

5:21

don't in fact I don't know what that

5:22

does. I just assume that what it does is

5:25

it somehow changes either implementation

5:28

or how optimizations work to the point

5:30

where reading becomes much more of an

5:31

optimize and writes become more

5:33

penalized. In other words, this is

5:34

something that's read a whole bunch. As

5:36

it says right here, read mostly, right?

5:37

It just seems pretty straightforward.

5:39

You can change data structures or you

5:41

can change optimizations to make these

5:42

things work where reading is the kind of

5:44

the expected path. And there's actually

5:46

quite a bit of this throughout this

5:48

entire uh little argument going back and

5:51

forth of the people who reviewed the

5:52

code. And it's because they signed their

5:54

name, but they missed this bug. And

5:57

yeah, the bug wasn't a big deal. Yeah,

5:58

it was fixed. But it's kind of this

6:00

weird cope that I'm seeing that yeah, I

6:03

would have reviewed harder knowing it's

6:04

an AI. I I don't I don't really

6:06

understand that. I would just assume you

6:08

review well because that's what you got

6:10

to do. And you know, I hate to be on the

6:12

side of defending anything with AI, but

6:14

this just feels like something I have

6:16

to. The outcome of after 6 months of

6:18

arguing is this document right here, AI

6:20

coding assistance, which effectively

6:22

says that, hey, if you're going to be

6:24

contributing to the Linux kernel using

6:26

an LLM, you have a couple requirements

6:28

yourself. All code must be compatible

6:30

with the licenses. And B, AI agents must

6:32

not add sign off by tags. They can only

6:35

add an assisted by tag. That means

6:38

somebody else is liable. Which honestly

6:40

is a pretty good approach. Meaning that

6:42

any line of code that hits the potential

6:45

to become merged into the kernel,

6:47

somebody has to say, "Hey, I own this

6:49

code. Hey, I fully understand it. I've

6:51

done 100% of the review and I say this

6:54

code is good." You cannot allow a

6:56

machine to say the code is good. But a

6:58

more interesting point in this argument

6:59

is actually down below in one of the

7:01

many comments that says, "At the same

7:02

time, the explicit goal of generating

7:04

code with LMS is to make every developer

7:06

more productive at writing patches,

7:08

meaning there will be more patches to

7:09

review and reviewers will be under more

7:11

pressure. And in the long term, there

7:13

will be fewer new reviewers because none

7:14

of the junior developers who outsource

7:16

their understanding of the code to an

7:18

LLM will be learning enough to take on

7:20

that role." It's actually kind of an

7:22

interesting uh proposition. And it's

7:25

definitely something I've talked about a

7:26

whole bunch, which is the value of

7:28

learning in the day and age of AI. Do

7:31

you really need to learn? Should you

7:32

really take the time to understand

7:34

everything or should you just like, yo,

7:36

Dario, take the wheel. And for me, you

7:39

know what? This is my culture, okay?

7:42

It's learning. And for a juniors out

7:44

there, when they see that and they hear

7:45

that, they're like, that that's your

7:46

culture learning. I really do have to

7:48

sympathize with this, though. right at

7:50

the very end right here says, "I think

7:52

writing code is already the easiest and

7:54

most enjoyable part of software

7:55

development." So, it seems like the

7:57

worst part is trying to be automated

7:58

away. Honestly, I can feel that so deep.

8:01

Holy cow. The most enjoyable part of the

8:04

software development practice isn't

8:06

trying to figure out what to program.

8:08

That is long. That is difficult. And it

8:10

takes a whole bunch of time to come up

8:12

with good ideas. I can tell you I have

8:14

failed. I have created so many projects

8:16

that absolutely went to the wayside.

8:18

I've created few projects that actually

8:20

were a very, you know, high amounts of

8:22

value and people ended up using a whole

8:24

bunch. But the coding part, the coding

8:26

part is and has always been the easiest

8:29

and the most enjoyable part of software

8:31

development. So, I can like I completely

8:34

sympathize with this. This has been one

8:35

of my big gripes about just reviewing

8:37

code. Reviewing code ain't fun. Nobody

8:39

likes reviewing code. And this is why

8:40

ultimately I think this little document

8:42

that's been developed, this is just a

8:44

stop gap. I don't know how long this

8:46

document can uphold, but my guess is it

8:49

won't uphold that long. Meaning that

8:51

they're going to claim the new tools are

8:53

able to review just as well as humans.

8:55

And it's going to kind of go into this

8:56

point of why can't I have agents on the

8:58

signed off tag? Hey, they're better at

9:00

code reviewing than I am. They find more

9:02

bugs than I do. And so my guess is

9:04

within one to two years this is going to

9:07

have to change because the amount of

9:09

patches coming in could very well not

9:11

explode by like double but could end up

9:13

10xing in the amount of reviewers are

9:16

simply going to be unable to get through

9:18

that code and some other thing is going

9:21

to have to happen. I somehow doubt that

9:23

Lionus is gonna allow Linux to be

9:25

sloppified, but the pressure from the

9:28

slop trebuche, it's intense. And my

9:31

suspicion is that it's only going to

9:32

grow year over year. You probably

9:34

noticed that I actually haven't said

9:36

much on whether I think you should be

9:38

able to use LLMs or not inside the

9:40

kernel. And that's because honestly, I

9:41

don't have a really good gauge. That

9:43

patch that we looked at, it was pretty

9:45

well done. Yeah, the read mostly thing

9:47

should have been caught. We could blame

9:49

the reviewers. We could blame the person

9:51

who created Sash 11 for not catching it.

9:54

But mistakes are made and it probably

9:56

would have even been not caught by an AI

9:58

as well potentially, but I'm also on the

9:59

other side of the fence, which is that I

10:01

think you should review the code you put

10:04

out there. Like I'm still on that side.

10:06

And so I don't know what it's going to

10:07

take for me to change that. And so even

10:09

if I can produce infinite amount of

10:12

code, it also means that I can't go

10:14

infinitely fast because I still have to

10:17

read it. And my guess is that Linux will

10:19

be in this kind of state for a very very

10:21

long time. And of course very long in

10:23

the day and age of technology is

10:25

somewhere between 2 to 50 years. I don't

10:27

know. Okay. The future's difficult.

10:29

That's it. That's what I wanted to yap

10:30

about. I always find the Linux, you

10:32

know, the Linux mailing list and the LWN

10:35

just very interesting. this kind of

10:37

argument that's always happening that

10:39

these are people that are deeply

10:40

concerned about code, the code craft,

10:42

the process of creating code, and kind

10:45

of their gigantic responsibility to

10:47

never be wrong. So for me, this is kind

10:49

of like a large, I guess, canary in the

10:51

coal mine for AI usage. If the kernel

10:54

can remain as consistent as it has been,

10:56

unlike all these sloppied websites, and

10:59

be able to take advantage of using AI,

11:01

then hey, you know, it's probably means

11:03

me and you need to take more looks into

11:05

it all. And and most of all like I think

11:07

the big thing to just always think about

11:08

is that at the end of the day writing

11:10

code and building things and building

11:12

kind of your ideas like this is amazing.

11:14

This is an amazing thing we get to do.

11:17

Who would have guessed in the entire

11:19

history of the world there would be

11:21

something in which millions of people

11:23

could create whatever comes to their

11:25

mind. We live in truly kind of like an

11:27

amazing day and age. So really with all

11:30

the FUD with all the kind of just

11:32

non-stop fear-mongering. Sorry, I keep

11:34

putting this. I just I just love taking

11:36

drinks. I just leave it right in the

11:37

camera shot. So, with all the

11:39

fear-mongering being done by Daario and

11:41

Sam and these hype beasts, to me, I look

11:43

at it as one of the, you know, one of

11:45

the greatest days to be writing software

11:47

and to have expertise because expertise

11:50

probably going to be needed for quite

11:51

some time. So, sorry. I know some of you

11:53

want her to G on your stack until you

11:55

produce 37,000 lines of code, but for

11:57

the rest of us, we're still just trying

11:59

to make sense of the world. Hey, the

12:00

name, it's the hopeen, okay? That's what

12:03

I want. That's what I want in life.

12:05

Also, I do think there's a big issue

12:06

with copyright and I really do hope some

12:08

people go to jail because honestly, the

12:10

fact what they did to Aaron Schwarz just

12:12

a little bit earlier, only a few years

12:14

back, but somehow all the execs and all

12:17

the researchers just stealing massive

12:19

amounts of stuff face absolutely no jail

12:21

time or consequences seems a little bit

12:23

suspicious to me. Just throwing it out

12:25

there. If we were to get to true

12:27

justice, I think jail time is in order,

12:30

at least a court hearing.

12:32

A gen again again

12:34

a genen. Hey, do you want to learn how

12:37

to code? Do you want to become a better

12:39

back-end engineer? Well, you got to

12:40

check out boot.dev. Now, I personally

12:42

have made a couple courses from them. I

12:44

have live walkthroughs free available on

12:46

YouTube of the whole course. Everything

12:48

on boot.dev you can go through for free.

12:51

But if you want the gamified experience,

12:53

the tracking of your learning and all

12:54

that, then you got to pay up the money.

12:56

But hey, go check them out. It's

12:57

awesome. Many content creators you know

12:59

and you like make courses there.

13:01

boot.dev. dev/prime for 25% off.

Interactive Summary

The video discusses the debate surrounding the use of Large Language Models (LLMs) in Linux kernel development. It explores a specific instance where an AI-generated patch was merged without disclosure, leading to a performance regression and a broader discussion on liability, the necessity of code review, and the potential impact of AI on learning and the future of software development.

Suggested questions

3 ready-made prompts