HomeVideos

I Tested Kimi K3 So You Don't Have To...

Now Playing

I Tested Kimi K3 So You Don't Have To...

Transcript

293 segments

0:00

Boy, oh, boy, oh, boy, has a lot changed

0:02

in the last few hours. Kimmy K3 dropped

0:05

and it is completely upset both the

0:08

balance of power with closed source

0:10

models, as well as the balance of power

0:12

between the American sort of hegemony

0:15

model developers and the Chinese. So,

0:18

this model company, Moonshot, has been

0:20

working on Kimmy for a while now. And

0:23

I've talked about it a little bit on the

0:24

channel. You know, they've released them

0:26

open source on platforms like Ollama.

0:28

But this really takes the cake. I mean,

0:31

as you see here, when they dropped just

0:33

a few hours ago, Kimmy K3 is almost band

0:36

for band on deep software engineering.

0:39

I'd very confidently say this is now in

0:42

frontier model territory. Frontier

0:44

software engineering, it actually whoops

0:46

GPT 5.6 souls' ass. Their own internal

0:49

benchmarks, as biased as they may be,

0:51

tell a similar story. And a few of these

0:54

terminal-style benchmarks, like Terminal

0:56

Bench 2.1, show it better even than

0:59

Fable. In this case, Program Bench far

1:02

better than Fable. So, this is going to

1:03

change the, you know, balance of user

1:06

preferences quite a bit as well. Because

1:08

why would you pay Fable prices when you

1:10

can get, you know, 98% now of Fable 5

1:13

intelligence at Sonnet 5 costs? And

1:16

despite how most American companies and

1:19

wider companies feel about giving a

1:21

bunch of data to, you know, {quote}

1:22

{unquote} scary Chinese models, there's

1:25

a lot to be said. I mean, if they pump

1:27

this thing out into the open source

1:29

ecosystem like they did with Kimmy K 2.7

1:31

code, that's significantly going to

1:33

change the economic calculus of

1:34

something like Anthropic, uh, OpenAI,

1:37

and so on. So, like, I'm going to try

1:38

these models out right now with you

1:40

guys, so you could see just what it

1:42

means when we're not staring at isolated

1:44

benchmarks. But I did want to be clear,

1:46

this is a major upset. Like a lot of

1:48

people in MakerSchool, a lot of people

1:51

in the companies that I consult with,

1:52

they've already asked me stuff like,

1:54

"Nick, what would it take to move my

1:56

entire infrastructure over to Kimmy K3

1:58

versus continue operating it on, you

2:00

know, closed sort of locked vendors like

2:02

OpenAI and Anthropic. And some of these

2:04

are billion-dollar companies. I mean,

2:05

this is not a small feat. All right, so

2:08

first we have some pretty basic design.

2:10

This is a simple state of remote work

2:14

sort of research paper. As you can see,

2:16

the designs now are pretty similar. And

2:18

there's a couple of reasons for that.

2:20

Odds are Kimmy K3 distilled from Fable 5

2:23

directly or there was a fair amount of

2:25

sort of like cross-contamination of

2:26

datasets and so on and so forth going

2:28

on. I think um they're also doing a lot

2:31

of like opinionated training to try and

2:33

get the models to do things like vary

2:34

font styles and so on and so forth. So,

2:36

nothing really exciting here. Obviously,

2:38

the quality of Kimmy K3 is going to be

2:41

pretty similar to that of Fable. You can

2:43

see the same thing happening here on

2:45

like this little electric vehicle one.

2:47

Fable decided to use sort of a bland

2:49

research report for a sleep review,

2:51

whereas Kimmy over here took a little

2:54

bit more creative of a license. But you

2:56

can see that they still did something

2:58

very similar. They used sort of like

2:59

this old-school book style first letter

3:02

really, really big. I'd actually say

3:04

Kimmy probably sized that better, which

3:06

is interesting. Okay, now we have some

3:07

3D and WebGL scenes, and this is where

3:10

the qualities really start to diverge.

3:12

So, on the right-hand side here, this is

3:14

sort of like an orbital traffic grid

3:16

designed by Fable. So, you guys can see,

3:18

you know, it did it did the job. It is

3:19

technically a globe and it has

3:21

information sort of transferring from

3:23

one side to the next. It's kind of neat.

3:25

Uh however, if you compare that with

3:26

Kimmy's, you'll find a world of

3:28

difference. I mean, this is just not

3:30

even comparable, not even close. I can

3:32

still zoom in. I can still do more or

3:34

less everything, but, you know, we have

3:35

this extremely like, I don't know,

3:38

otherworldly sort of vibe. It's also a

3:40

whole country with like information

3:41

highways clearly spread throughout. And

3:43

as I zoom in, things get a little bit

3:45

clearer. You can see it's a

3:46

high-quality, high-resolution map with

3:48

this overlaid. I can't say the same for

3:50

this. So, you know, keep in mind these

3:52

are one-shots, obviously, but uh I would

3:53

say this did on par if not far better.

3:55

All right, now this is a Fables 5

3:57

endless flyover of some map that's being

4:00

procedurally generated um on the

4:02

right-hand side, and you know, like it's

4:03

pretty cool. There's some interesting

4:05

rumbling going on, which kind of scares

4:06

me, if I'm being honest. Uh you know, I

4:08

can do the same thing with

4:10

uh Kimmy K3 over here. I'd say the

4:12

landscape is a lot cleaner and a lot

4:14

prettier, probably a lot more beautiful.

4:16

That said, there's some issues with

4:17

reading the font up top. I can change

4:19

the speed by scrolling around. I'm not

4:21

going to scroll too much just cuz I want

4:22

to show you guys more. If I scroll down

4:24

here, we're now sort of doing a

4:26

particle-style galaxy, a a nebula. And

4:29

on the right-hand side here is Fables.

4:31

As you could see, most of the screen

4:32

just turns white when I go down here. I

4:34

do have the ability to move this. This

4:35

kind of reminds me of Spore. I don't

4:37

know if you guys ever used to play

4:38

Spore, but uh I used to play a lot of

4:40

that.

4:41

On the right-hand side here, uh looks

4:42

like we have the ability to adjust the

4:44

slider and uh you know, the the

4:46

brightness of the stars and stuff like

4:48

that myself, which is kind of cool. If I

4:49

click on this plus button, you can see I

4:51

can change the the amount of stardust,

4:53

the number of arms, you know, the spin,

4:56

and so on and so forth, the the the the

4:58

chaos marker, which is kind of cool. So,

5:00

if I exit that out, I mean, like this is

5:02

this is really nice. I don't know if I

5:03

can do that here. I don't think I can on

5:05

Fables' version. So, yeah, I mean, the

5:07

Aura definitely kicked Nebula Andraste's

5:10

ass on that one. I'm going to give Kimmy

5:12

K3 the win, but uh worth noting that did

5:15

take three times the amount of time that

5:16

Fable did. Okay, and then we have a

5:18

couple more mini-games on the right. We

5:20

have Fables 5 with uh you know, the

5:22

stacker. So, this is me stacking a

5:25

bunch. Looks pretty good. So, obviously,

5:27

I'm not too hot at that game, which is

5:29

how I lost. Over here, just got to close

5:32

some tabs before my computer uh has a

5:34

hernia.

5:35

Um you know, we could see the stacking

5:37

is like three-dimensional, which is kind

5:38

of cool. I like that. Um I would say

5:40

this is far more akin to the sorts of

5:42

things you'll see at like uh I don't

5:43

know, like one of those casinos or rec

5:45

rooms, which is kind of neat. Not really

5:47

seeing too many differences here in the

5:49

quality. Obviously, one model is a

5:51

little bit better at something, another

5:52

model is a little bit better at

5:53

something else. The point is, one is at

5:55

Sonnet 5 pricing, and the other one is

5:57

at Fable 5 pricing. So, sort of like,

5:59

choose your own mini game. Would I

6:00

rather spend two times longer for, you

6:02

know, a third or a fifth of the price,

6:04

or rather get it done in 80 seconds as

6:06

opposed to 240 seconds, but sort of burn

6:08

a hole in my wallet? What does the world

6:10

look like when we all have access to

6:12

Fable 5 intelligence in our pockets for

6:15

virtually free? Cuz if it's not clear,

6:18

that's where all of this stuff is going.

6:20

The intelligence isn't getting more

6:21

expensive. Bar for bar, pound for pound,

6:24

it's dropping something like 30 to 40

6:27

times a year, uh based off of my

6:29

calculation. It is getting so

6:31

overwhelmingly expensive that I see it

6:33

feasible at some point in the future

6:35

that you actually have large language

6:36

models running uh on headphones capable

6:39

of doing the vast majority of the

6:41

lifting that Fable 5 is doing. And

6:42

that's just in a few years. So, just for

6:45

the sake of argument, what do you think

6:47

would happen in a world like that? Now,

6:49

I don't think it takes a neuroscientist

6:51

to explain this, but uh I have a

6:53

published paper in neuroscience, so I'll

6:55

try and tell you. To make a long story

6:56

short, we'll have significantly

6:58

democratized access to an average human

7:01

intelligence. We'll be able to spin up

7:03

dozens, if not hundreds, of these

7:05

literally at the snap of a finger to

7:07

help us with any intellectually valuable

7:09

task. When this occurs, the entire

7:12

scarcity aspect of the economy will

7:14

change completely. I mean, right now,

7:16

human knowledge work is the bottleneck

7:18

to virtually everything. You know, if

7:20

you want a service, you want a website,

7:21

if you want a consulting report, if you

7:23

want to develop a new piece of

7:25

technology, if you want to do material

7:27

science, push the frontier of science,

7:29

and so on and so forth, forward, you

7:31

need people, right? Like, we're people.

7:32

It's our meat brains that are doing all

7:34

of this. What does the world uh do? How

7:37

do we value economic and human economic

7:40

labor, specifically? What happens to the

7:42

vast majority of jobs that we all have?

7:45

So, an interesting thought experiment

7:46

for you is let's say for whatever

7:49

reason, maybe just the laws of physics,

7:51

models stop growing smarter. They just

7:53

get no smarter starting right now. So,

7:55

basically we capped out at, you know, a

7:57

Fable 5 consumer level model

8:00

and we just can't grow it smarter.

8:02

What do you think happens to the world

8:04

over the course of the next 20 or 30

8:06

years?

8:07

Does all scientific advancement and

8:09

progression and development stop even

8:11

though the models themselves are no

8:13

longer going to grow any more

8:14

intelligent? Human beings will come up

8:16

with ways to use these models, to

8:19

incorporate them into society, to weave

8:21

them into the fabric of social life

8:23

tremendously over the course of the next

8:25

20 or 30 years. I mean, we'll have AI

8:27

assistants waking us up in the morning.

8:29

We'll have AI assistants coordinating

8:31

the the building and architecture of new

8:33

freaking planets. We could have a Fable

8:35

5 level intelligence shepherd thousands

8:37

of humans across a freaking generation

8:39

ship in the year 2982.

8:42

The model intelligence really isn't the

8:44

bottleneck anymore if that makes sense.

8:45

It's the distribution. So,

8:48

what Kimmy here, a Kimmy K3 effectively

8:50

solves is that distribution. You could

8:52

have everybody on planet Earth, you

8:54

know, who has the GPU access, which is

8:56

another story entirely. But everybody on

8:58

planet Earth running this on a future,

9:00

you know, Olama style setup where it's

9:02

totally open. And to be clear, I don't

9:03

have the answers to these questions. I'm

9:05

just asking them because I think now is

9:06

the time to ask them, not in 10 years

9:08

when those answers could have helped us

9:11

10 years ago, right? So, [snorts]

9:14

seeing cool model technologies like

9:16

Kimmy K3 really fires me up and I'm

9:18

happy because I don't want like a

9:20

monopolar

9:22

uh monopolar sort of style future where

9:25

all of the intelligences congregated and

9:27

segregated in one country. I think we

9:28

should naturally have some form of

9:30

competition. Anyway, think about it.

9:32

We're all going to have Kimmy K3 style

9:34

models on our phones. They're all going

9:35

to be 100% accessible, crazy uptime.

9:38

What more do you really need at our

9:40

current level? And if the answer to that

9:42

question is nothing, then clearly

9:43

something about our current level is

9:45

going to have to change. There's just no

9:47

way we can sustain this current economy

9:49

in a future where models do vast

9:51

majority of everything.

Interactive Summary

The video discusses the release of Moonshot's new model, Kimmy K3, which challenges the dominance of major closed-source AI models like Fable 5, particularly in software engineering. The creator showcases the model's capabilities in various tasks such as WebGL, 3D scenes, and mini-games, highlighting that Kimmy K3 offers comparable or superior performance at a fraction of the cost. The video concludes with a broader reflection on how the democratization of high-level AI intelligence will significantly reshape the global economy and human labor, moving from a bottleneck of intelligence to a challenge of distribution.

Suggested questions

2 ready-made prompts