HomeVideos

I Deleted All My Claude Skills... And Claude Got Smarter

Now Playing

I Deleted All My Claude Skills... And Claude Got Smarter

Transcript

404 segments

0:00

So, I listened to the creator of Claude

0:01

Code explain why Claude has gotten worse

0:03

and how to actually fix that. And I

0:04

learned something that sounds really

0:06

weird, but more skills and more system

0:07

prompts are probably breaking your

0:09

system. So, I dug deeper into this and I

0:10

started doing some testing and I found

0:12

that there are three super important

0:13

things that everyone has to be doing

0:14

right now. It's super easy to do and you

0:17

don't need any technical experience to

0:18

actually implement this stuff. So, I

0:20

don't want to waste any time. Let's just

0:21

get straight into the video. So, let's

0:22

just start off with some real quick

0:23

context on what they're actually doing

0:24

behind the scenes at Enthropic.

0:26

>> Let's talk a bit about the new release.

0:29

You actually deleted over 80% of the

0:32

system prompt from claw code.

0:35

>> Yes.

0:35

>> Tell us more about that.

0:36

>> I think something that a lot of people

0:38

might not realize is um clawed code as a

0:42

product and as a harness is just always

0:44

changing. We're always adding stuff.

0:46

We're always deleting stuff. Every time

0:49

that a new model comes out, we delete a

0:51

bunch of the system prompt. Change a

0:53

bunch of the system prompt. We change

0:55

the set of tools all the time. We change

0:57

the prompts for the tools all the time.

0:59

And the reason is every model is very

1:03

different. So something that you did for

1:06

one model maybe three months ago, it

1:08

just might not translate at all to the

1:11

next model. And so one thing about Opus

1:13

5 is it's just really intelligent. And a

1:17

lot of the stuff in the system prompt

1:19

was correcting for these behaviors that

1:21

the model should have known, but uh it

1:24

didn't. Now, Opus 5 just does it.

1:26

>> So, that's interesting. It makes you

1:27

think a little bit, but I just wanted to

1:29

sort of lay the foundation of they're

1:31

doing that adanthropic. They are

1:33

treating each different model as like a

1:34

different completely different

1:36

personality, different flaws, different

1:37

strengths, and different ways that you

1:38

should prompt it. And so, whenever a new

1:40

model drops, one of the things that I

1:41

tell you guys is like, hey, run this

1:43

model through your skills. Make sure

1:45

they all still work. Make sure they all

1:46

still feel good. And for some reason

1:47

with Opus 5, it's just felt degraded to

1:49

the point where I'm actually going back

1:50

to 4.8 every once in a while when I need

1:52

to do something and I'm just getting

1:53

better results. But I never really

1:55

thought about, hm, maybe if I just got

1:57

rid of some of my instructions and got

1:59

out of the model's way, it is actually

2:01

just more intelligent now and it could

2:02

handle those things. So take a look at

2:04

this next clip.

2:04

>> Yeah. And and for people that aren't

2:06

building aic products, but you're using

2:08

cloth code every six months, delete your

2:10

quadd, delete your skills, delete your

2:13

hooks, see what the model does, and it

2:15

might surprise you. And actually for

2:17

Opus 5, this is something we really do

2:20

recommend is just try deleting all of

2:22

these things because the model might

2:25

really just not need all those

2:27

instructions that you needed for past

2:28

models.

2:28

>> So when you think about that for a sec,

2:31

I feel like that almost sounds

2:32

counterintuitive because what we've been

2:33

told is, hey, you know, if everyone has

2:34

access to the same model or some this is

2:36

something that I say a lot, then

2:37

wouldn't everybody be getting the same

2:39

results? And to an extent that is very

2:40

true. But if Boris is basically saying,

2:42

hey, remove your skills, remove your own

2:44

system prompts, and just see what's

2:45

possible. It's a different way to think

2:47

about it because the the the immediate

2:49

thing that I think about is like, okay,

2:51

let's say that I was teaching um a

2:53

10-year-old, a college student, and

2:55

someone that's been in the workforce for

2:57

10 years how to do something, how to do

2:58

something like create a slide deck. I

3:00

would probably give the 10-year-old the

3:02

most specific instructions. I would say,

3:03

"Hey, here's exactly what you should be

3:05

looking for. Here are the 10 slides to

3:07

make. Here's what the title should look

3:08

like. Here's the color. Here's the way

3:09

that the text should be formatted." I

3:11

would be super specific and I basically

3:12

wouldn't give that 10-year-old any

3:15

freedom really to just kind of go

3:16

experiment because if it's a high stakes

3:18

thing, we want to just steer it. But as

3:20

you work your way up the chain in, I

3:21

guess like intelligence and experience,

3:23

what if you gave that same instruction

3:25

set to the person who's been working for

3:27

10 years and has made hundreds of slide

3:29

decks, you're going to get in their way.

3:31

They're not going to be able to actually

3:32

use their subject matter expertise and

3:34

their intelligence to put their own spin

3:35

on it. And so I'm not saying that it's

3:37

always the case, but I think that there

3:39

is an element of like removing some of

3:41

the some of the like kind of

3:42

instructions and constraints on a model.

3:44

They call this unhobbling the model,

3:45

which I'll play a clip from in just a

3:47

minute here. But take a look at this

3:48

example I ran. So after I watched this

3:49

interview, I made a duplicate of my repo

3:51

and I took away the cloud MD and I took

3:53

away all my skills and I just started

3:54

using it a little bit to see how it

3:56

worked and it was okay. But what my gut

3:58

is telling me about that is when we're

3:59

doing something like we have, you know,

4:01

um, an AI operating system and we have

4:03

the cloud MD which is kind of routing

4:04

around to, hey, here is Nate's business

4:07

context. Here are where all these files

4:08

live. Here are where the wiks live.

4:09

That's important. We still need that

4:11

context of, hey, Mr. Cloud Code, here's

4:14

where you find your things. But when it

4:15

comes to actual tasks, maybe you just

4:17

get out of the way a little bit and

4:18

that's where your skills aren't as

4:20

important or your skills are changed.

4:21

And so what I mean by this is take a

4:23

look. Here is an output I ran where I

4:25

took this YouTube video, this uh

4:26

interview that we're watching. I gave

4:28

that URL to Claude and say, "Hey, make

4:30

me a YouTube resource guide, like the

4:32

resource guides that I make for you guys

4:33

when I have other videos." And so, what

4:35

this did is it broke it down, right? It

4:37

it created a nine-page resource guide.

4:39

It's formatted nicely. It has like these

4:41

colors. It has like these little blocks

4:43

and it has my also like my header,

4:45

right? It's got that header image. It's

4:46

got my YouTube channel linked right here

4:48

at the bottom. It links to AIS Plus. So,

4:50

those are things that are like very

4:51

specific. So, those are things that it's

4:53

like my preferences and my styles. Now,

4:55

look at this version. This is one on a

4:57

fresh session where I gave it basically

4:59

the same prompts, but it had no skills

5:00

or no context to work from. And here's

5:02

what it created. Obviously, it doesn't

5:04

look as good, right? It just looks a bit

5:05

messier. It doesn't have nice

5:07

formatting. There's no header. But I

5:09

think that this is a better output

5:11

because I didn't tell it specifically,

5:12

hey, for a resource guide, do this,

5:14

this, and this. Here's how you break it

5:15

down. This actually broke this down into

5:17

ideas, which I thought was really nice.

5:18

And each idea has a timestamp. So here

5:21

we have prompts are disposable. Here we

5:22

have rebuild from observation. All of

5:24

these main ideas that it covers, we have

5:27

timestamps. So arguably I like the

5:29

content of this version better which had

5:31

no skills, no clawmd. But what I would

5:34

do is I would bake this into a skill and

5:35

say, "Hey, by the way, make the resource

5:37

guide however you want, however you see

5:39

fit. But what I want you to do is take

5:41

this image and put it in the header and

5:42

link to my YouTube channel at the top

5:43

and link to AIS Plus at the bottom." So

5:45

those are the areas where I think it's

5:47

less in my mind about deleting all your

5:48

skills. It's more about really thinking

5:51

about them and maybe making versions of

5:53

them that aren't as specific. And real

5:55

quick, as I'm editing this video,

5:56

there's one other thing I wanted to

5:57

throw out there that I think is super

5:58

important to think about, which is you

6:00

can't just take advice you see from

6:02

Boris Churnney or from some other

6:03

YouTuber or from me or from someone in

6:04

the community and just blanket statement

6:07

apply it to your work. It just doesn't

6:09

work like that. Because what my gut's

6:10

telling me is a lot of this advice that

6:12

Boris is going to tell you or that you

6:13

might see Andre Carpathy tweet about. A

6:15

lot of their advice is coming from the

6:17

perspective of someone who's like in the

6:19

weeds designing harnesses, working on

6:20

massive code bases every day training

6:23

models. That's not what I do. I don't

6:25

know if that's what you do or not, but

6:26

you should probably be taking advice

6:28

from people who are using the AI systems

6:30

the same way you want to. For me, that's

6:31

generally bunch of research, knowledge,

6:33

work, um, creating documents, creating

6:36

deliverables, but not as much like

6:38

building software. that's not what I'm

6:40

doing on the day-to-day. So, I just

6:41

wanted to throw that out there. I think

6:42

that Boris has some really good advice

6:43

here, but the reason why I didn't

6:45

actually go out and sweep delete all of

6:46

my stuff is because based on my findings

6:49

and my testings, that wouldn't be as

6:51

valuable to me as just kind of like

6:52

reworking them. I think people that are

6:54

building products and software have like

6:55

these all of these skills to like build

6:57

and to orchestrate. And maybe that's

6:59

where those skills should be completely

7:00

deleted because the model and the

7:02

harness just does that better now. But,

7:03

as you saw from this demo, like I still

7:05

need the skills to say, "Hey, you're

7:06

going to grab this image. You're going

7:07

to put that there. um you're going to

7:09

use these color schemes because that's

7:10

like our brand guidelines. That sort of

7:12

stuff is still important context for my

7:14

systems for my knowledge work. So just

7:16

something important to keep in mind. And

7:18

Boris seems to completely agree. Let me

7:20

show you guys this next clip.

7:21

>> Yeah. So hobling is this idea in a

7:23

research that the model is doing

7:25

something and you're just getting in the

7:27

way

7:29

there. There's this kind of like way of

7:31

thinking about it that I really like.

7:32

It's very useful when you're building

7:34

product and um it's called product

7:37

overhang and the idea is

7:41

the model is able to do all sorts of

7:44

things with today's models not a future

7:47

model but today's model that we have not

7:50

yet realized

7:52

and there are so many capabilities the

7:54

model has like this that people are not

7:57

aware of so there's a couple things that

7:59

I will think about one is

8:03

you should give the model slightly

8:06

harder tasks than what you think it can

8:08

do. I think a a really common mistake

8:11

that I see is people are using cloud

8:13

code. They're using cloud and they they

8:16

just give it like way overly specific

8:18

instructions. They're like, I want you

8:19

to do this, but I want you to do it in

8:21

this way, this way, this way. You must

8:22

do like one, then two, then three, then

8:23

four. And for modern models, that's

8:26

actually really not the way to do it.

8:28

You want to go a little bit higher

8:29

level. You want to describe the task,

8:31

you want to describe the guardrails, you

8:33

want to describe like the exit criteria

8:34

and then just go with the model cook and

8:38

come back in a little bit and I think

8:40

it'll it'll surprise you like and again

8:42

like this is just not something that

8:43

would have worked 6 months ago but it

8:44

does work today.

8:46

>> So if you weren't convinced when I said

8:47

that earlier hopefully you're convinced

8:48

now that you've heard Boris Churnney say

8:50

that and I think that there's so many

8:51

things that go into this right there's

8:53

obviously the balance of using your

8:55

closet MD in a way that doesn't hobble

8:57

the model. Same thing with your skills.

8:59

And then there's also the balance of

9:00

actually getting an output that you

9:01

like. Because I think a lot of us get

9:03

frustrated when we ask cloud to do

9:04

something, it comes back and the

9:05

output's just very mediocre. But maybe

9:07

that was on you. Maybe it was your job

9:09

to say, hey, here is what good looks

9:11

like. Because if it doesn't know what

9:12

good looks like, how does it actually

9:13

create something that's good? So like

9:15

Boris said, you give it a highle goal.

9:17

You set the standard and say basically,

9:19

don't stop until you're done, until

9:21

you've hit this standard. So when you

9:22

think of setting a SL goal when you're

9:23

using these AI agents, you're not just

9:25

saying, hey, here's what good looks

9:27

like. You're also saying here are X, Y,

9:28

and Z things that you do in order to

9:30

prove that this has been reached. And

9:33

you keep iterating on that until you've

9:35

actually proven it. You verify yourself

9:36

so that I don't have to verify. For

9:38

example, something that I've been

9:39

throwing at the end of my SL goals after

9:40

I've given it all the standards and the

9:42

verification methods. I basically say

9:43

like, I'm not looking for a prototype or

9:45

a proof of concept. I'm looking for you

9:47

to give me something that has been

9:48

tested and iterated on 10 times and has

9:50

been fully QA and that's ready to go to

9:52

market tomorrow. That makes it a little

9:53

bit more emotional. And sometimes when I

9:55

prompt things with emotion, it seems to

9:56

do better. But let's take a look at what

9:58

Boris said about verification.

9:59

>> I I think the skill nowadays is less

10:02

about prompt engineering and more about

10:05

figuring out how do you give claude a

10:07

hard task? That seems a little bit too

10:10

hard. And then how do you make it

10:12

possible for cloud to verify its work

10:14

along the way? And the verification I

10:17

think is probably the single most

10:18

important thing that people do not get

10:20

right. It's really really interesting to

10:22

see how the creator of Claude Code is

10:25

thinking about this because the more

10:26

that I get into the space and the more

10:28

that I talk to people that are starting

10:29

to get into the space, the more I

10:31

realize that the skills that are

10:34

transferring over here, it obviously

10:36

doesn't matter. They're tool agnostic.

10:37

If you want to use cloud code, Hermes,

10:38

agent code, whatever it is, the tools or

10:40

the skills that really are important is

10:42

managing AI agents. And when I say

10:44

managing, I mean a similar way to when

10:45

you manage people. Now, I haven't been

10:47

managing people for very long, right?

10:49

But what I do know is that good managers

10:51

don't just micromanage, right? They're

10:53

not saying, "This is what you should do.

10:54

This is how you do it. Go do it." They

10:56

get out of their people's way. They

10:58

check in with them. They review. They

11:00

have judgment. They have taste. But they

11:01

also let those people use their brains.

11:04

Like they were hired for a reason. And

11:05

so essentially, you should be thinking

11:06

about all these agents and all these

11:08

sessions that you're spinning up. You're

11:10

using them for a reason. Outsource the

11:12

thinking to them. Let them think and let

11:13

them come up with ideas, but give them

11:15

an ability to have other agents play

11:17

devil's advocate. give them an ability

11:19

to check their own work and verify.

11:20

Outsource some of the thinking, but

11:22

never outsource the understanding. At

11:23

the end of the day, you're still like

11:24

the founder of these agents and you are

11:27

driving the vision, but they're able to

11:28

help you so much. So, anyways, I could

11:30

talk about this all day. I think that

11:31

this is super interesting. If you guys

11:33

want to check out this interview, it was

11:34

like a 35minute interview on the Y

11:36

Cominator YouTube channel. I will leave

11:38

a link to it in the description as well.

11:39

These are just some of the key

11:40

highlights that I wanted to pull out and

11:42

break down. And let me know if you guys

11:44

enjoy this type of video, if you enjoy

11:45

this type of breakdown. And hopefully

11:46

you found it valuable, you learned

11:47

something new, and there's something

11:48

that you want to try. If you did, please

11:50

give it a like. It helps me out a ton.

11:51

And as always, I appreciate you guys

11:52

making it to the end of the video, and

11:54

I'll see you on the next

Interactive Summary

This video explores the counterintuitive concept of "unhobbling" AI models like Claude, where reducing overly specific system prompts and rigid instructions can lead to better performance. By treating AI models more like intelligent professionals rather than rigid task-executors, users can allow models to leverage their native capabilities more effectively. The video highlights the importance of providing high-level goals and clear verification criteria rather than micromanaging every step.

Suggested questions

3 ready-made prompts