HomeVideos

CLAUDE CODE ADVANCED FULL COURSE (3 HOURS)

Now Playing

CLAUDE CODE ADVANCED FULL COURSE (3 HOURS)

Transcript

6952 segments

0:00

Hey, this is the definitive Claude Code

0:01

course for advanced users. I use Claude

0:03

Code and AI agents in my own business

0:05

everyday to generate over $4 million a

0:07

year in profit. I also teach around

0:09

2,000 people how to use Claude Code and

0:10

other tools to improve their lives both

0:12

personally and business. Okay, so this

0:14

course is going to assume a foundation

0:16

of Claude Code experience. It's not for

0:18

total beginners, but if you are total

0:19

beginner and you happen to stumble on

0:21

this course, that's okay. Just look over

0:23

my left shoulder here, click that

0:24

button, and then I have a 4-hour guide

0:26

that will walk you through everything

0:27

you need to get to the point where you

0:28

understand what I'm about to say.

0:30

Assuming you're still here, no fluff,

0:31

here's what we're going to cover. Let's

0:32

start with an advanced look at

0:33

Claude.mds and system prompts, and learn

0:36

how to optimize these to actually

0:37

improve quality, which is simpler than

0:38

you think. We'll then cover agent

0:40

harnesses and how to build larger

0:41

projects with Claude Code. After that,

0:43

we'll chat agent teams and other

0:44

examples of extreme task

0:46

parallelization. Then we'll do skills,

0:48

sub-agents, and other forms of

0:49

organization. After that, I'll cover

0:51

Carpathy's auto research approach for

0:52

improving stuff progressively over time,

0:54

and a few actual use cases you can apply

0:56

this to, not just fancy demos. We'll

0:57

then talk browser automation, the major

0:59

players. We'll do computer use, browser

1:01

use, and which tools to apply to

1:03

different use cases depending on what

1:04

you want. I'll then cover how to deal

1:05

with performance fluctuations in Claude

1:07

Code because they do happen, as well as

1:09

some alternatives that you guys could

1:10

use and ways to bundle in multi-agent

1:12

orchestration into your workflow. We'll

1:13

then cover workspace organization, so

1:15

for personal, business, and then even

1:17

client projects assuming you're selling

1:18

the sort of thing as a service. Security

1:20

for larger projects, we'll chat stuff

1:21

like the recent auto mode. We'll talk a

1:23

little bit about OAuth. And at the end,

1:25

I'll finally round it out with a

1:26

discussion about where I think Claude

1:27

Code is going and the future of work

1:29

more generally. Hopefully you're as

1:30

excited as I am to level up your Claude

1:32

Code skills. Please use the bookmarks

1:33

and chapter headings as needed to jump

1:35

around the course. Subscribe to the

1:36

channel, and let's get into it.

1:39

So for most of the course, I'm going to

1:41

be building directly using the Claude

1:42

Code extension inside of Antigravity.

1:45

That's this over here. If you don't have

1:47

Antigravity installed, this isn't

1:49

installation tutorial, but get that from

1:51

Google's official antigravity.google

1:53

website. Then head over to to

1:56

click on Claude Code for VS Code, give

1:58

that an install, and then everywhere you

1:59

go, you'll have this little Claude logo

2:02

that you can use to spin things up.

2:03

After a brief login, you'll have more or

2:05

less the exact same layout that I do. I

2:07

want you to know, though, that the

2:08

Claude desktop app is also getting

2:10

better and better by the day. And

2:12

because Claude is attempting to get you,

2:13

obviously, on their infrastructure as

2:15

opposed to on your own, they're just

2:17

continuously adding new cool features

2:19

that allow you to do things like mobile

2:21

development and so on and so forth. So,

2:22

everything I'm going to show you today

2:23

works in both the Claude Code tab of the

2:26

Claude desktop app, also works natively

2:28

inside of a Claude Code's extension with

2:30

an anti-gravity or some other, you know,

2:32

IDE-like thing. So, if you're

2:33

intimidated at all by the way that I've

2:35

laid things out, what all these

2:36

different folders mean, and how they

2:37

collaborate in order to improve your

2:39

workflow, I'm going to cover all that in

2:40

this course. First, though, we're going

2:42

to cover claude.md and other advanced

2:44

system prompts. Basically, how to set up

2:46

your system prompts in a very efficient

2:48

and effective way, both to save you

2:50

financially, but also to improve the

2:52

quality of your outputs, and

2:53

significantly minimize the amount of

2:54

time it takes to build anything. So,

2:55

what is a claude.md really? Well, as far

2:58

as I could tell, it's four things. The

2:59

first is it's a form of knowledge

3:02

compression, okay? And when I say

3:03

knowledge compression, what I mean is,

3:06

instead of Claude having to read through

3:08

your entire workspace, you know, file by

3:11

file, like for instance, over here,

3:13

instead of it having to open up every

3:14

single folder here, every single one

3:17

here, read through all of the files and

3:19

so on and so forth, to be able to reason

3:21

and then make high-level declarations

3:22

about your code base or folder,

3:24

what your claude.md

3:26

okay, is it basically just compresses

3:28

all of that down into a highly succinct

3:30

summary of what the heck is going on in

3:33

your freaking folder. So, that the next

3:35

time you say, "Hey, what was that file I

3:37

made a couple of weeks ago about X, Y,

3:39

and Z?"

3:40

Claude doesn't have to look through

3:41

every single file in your code base, you

3:43

don't have to spend a tremendous amount

3:44

on tokens, and you also don't have to

3:46

wait a long time. It's just sort of

3:48

baked into the claude.md, or at least a

3:49

reference to where the file lives is

3:51

baked into the claude.md. Um so you can

3:53

actually like reason with it at a

3:54

superficial level, at a bird's eye

3:56

level, as opposed to actually going down

3:57

through the weeds.

3:58

So that's sort of like the very first

4:00

thing that I'd say, you know, a

4:01

Claude.md is.

4:03

The second thing that a Claude.md is is

4:05

it's obviously your own preferences as a

4:07

user.

4:08

And what you'll find is, you know, more

4:10

or less every time Anthropic updates

4:12

Claude code, you have better and better

4:15

baked-in native preferences and

4:17

conventions for things like, you know,

4:18

delivering you file paths or

4:20

how to deal with like documentation or

4:23

debugging or how to update itself and so

4:25

on and so forth. But obviously, Claude

4:27

code lags behind these preferences a

4:30

little bit

4:32

because they have to see what users are

4:33

actually using it for and and you know,

4:34

like they collect that information and

4:36

figure out what ways to make things more

4:37

effective. So if you're an advanced

4:38

user, as I am, you'll have a list of

4:40

these preferences and conventions that

4:42

improve your user experience. And

4:44

advanced users will always have just

4:46

some better preferences that kind of

4:47

adapt their own workflow as well as, you

4:49

know, programming conventions, um ways

4:51

to organize information, structures, and

4:52

and that sort of thing. Okay? So it's

4:54

both a form of knowledge compression,

4:55

but it's also preferences and

4:57

conventions that are not natively baked

4:58

in that you get to decide on.

5:01

The third thing that Claude.md is is

5:04

it's a declaration of capabilities.

5:06

Now, I don't know how many times this

5:08

has happened, but if you do not have a

5:10

substantiated enough Claude.md, and then

5:12

you have, let's say, a skill somewhere

5:14

in your your workspace where you have

5:15

just some knowledge sort of floating

5:17

around in a few files.

5:18

And you say, "Hey Claude, do XYZ thing

5:20

for me. Go, you know, find some

5:21

knowledge on XYZ person or go do some

5:23

research or you know, compile a plan

5:25

using XYZ framework."

5:27

Half the time, okay, if it's not in your

5:28

Claude.md, Claude will just look at you,

5:30

metaphorically obviously, it doesn't

5:31

have eyes yet. And it will say like,

5:33

"Oh, like I don't have a built-in way to

5:35

do this. Sorry, what were you referring

5:36

to? Do you want me to build something

5:38

from scratch? I'll happy to do it."

5:39

And this this sort of slow down loop is

5:41

completely unnecessary. And so what

5:43

Claude.md allows you to do is basically

5:44

allows you just to itemize, okay?

5:47

You know, everything your agent can

5:49

currently do within your workspace and

5:51

you can make that really clear. You

5:53

could say, "Hey, you currently have

5:54

access to this functionality. You can do

5:56

this. Hey,

5:57

you know, you can build a full step plan

5:59

that lasts 10 or 15 minutes and then

6:01

execute it on an autonomy." In fact,

6:02

that's my that's my preference or the

6:04

convention that we're using. You know,

6:05

you can call this API, you can call this

6:07

database, you can retrieve all this

6:08

information, you can act autonomously

6:09

using browsers and so on and so forth.

6:11

The reason why that's important is

6:12

because as agentic as Claude is,

6:15

hopefully we're we're all still on the

6:16

same page here

6:17

about this fact. Claude still lacks a

6:19

lot of agency, okay? If you ask it to,

6:22

you know, help you do something or if

6:23

you ask it how long it would take to do

6:24

something, it'll often significantly

6:26

underestimate or overestimate because

6:28

it's not really factoring in its own

6:29

agentic capabilities. Like I asked it

6:31

the other day, "Um hey, you know, how

6:33

long is would this XYZ thing take to

6:34

build?" And then it was like, "About 3

6:36

months or so because you would have to

6:37

build this, you'd have to build that,

6:38

you'd have to build that." And it's

6:39

obviously like, "No, I don't have to

6:41

build that. I'm asking you to build it.

6:42

You could build it in 5 seconds, so why

6:44

don't you just go ahead and do it?" Or,

6:45

you know, you're having it do some API

6:47

stuff and then it sends you a little

6:49

command line interface prompt and it's

6:50

like, "Hey, just pump this into the

6:51

terminal." It sort of needs reminders

6:53

that no, I don't have to do this. That's

6:55

why I'm asking you to do it. So, you can

6:56

actually do all of this stuff, Claude.

6:58

Declaring capabilities in this way,

6:59

whether it's your own personal like

7:01

tooling or workflows or whatever or it's

7:03

you know, Claude understanding that it

7:04

has the ability to do things that it

7:06

might not realize at first glance is

7:07

pretty important.

7:09

And then finally, the fourth thing that

7:10

a Claude in MD is is it's a log of

7:12

failures and successes. What I mean by

7:15

this is as you accumulate various files,

7:18

as you accumulate, you know, bits of

7:20

code through your project and stuff like

7:21

that, every single one of these things

7:23

is hard-won. You didn't get them for

7:25

free. Realistically, you spent tokens

7:27

and then your time, which are soon to be

7:29

two of the world's most valuable

7:30

resources. And so, because you spent all

7:33

this time and energy, it is more

7:35

efficient for you to take all of the

7:38

learnings basically from every single

7:39

piece of development or so, every single

7:41

action Claude does And then insert it in

7:43

its next system prompt, then just have

7:45

it restart kind of from scratch every

7:46

time. You know, viewed another way,

7:48

mathematically, if this is the total

7:50

space of all of the different possible

7:52

things that Claude could do when you

7:54

say, "Hey, do X."

7:57

What this log of failures and successes

7:59

is doing is it's basically carving out

8:01

big chunks of this theoretical solution

8:03

space and it's saying, "Hey, no, you you

8:06

don't do anything over here because

8:08

we've already tried all this stuff over

8:09

here." This kind of looks like a planet.

8:11

Meaning the only things that you can

8:13

actually try, the only things that you

8:14

should try are kind of the things that

8:15

exist in between.

8:16

Okay, so basically what this log of

8:18

failures and successes does

8:21

is it just allows you to immediately

8:24

cross out like 80% of all possible

8:27

things Claude could do cuz it knows it's

8:28

actually tried that in the past and then

8:30

in that way focus its time, effort, your

8:32

tokens, your money, and then your your

8:34

energy on the 20% that actually matters.

8:37

So, these four will exist in different

8:39

sections in your Claude MD. They'll also

8:41

exist at different levels, both global

8:43

and local. So, what I'm going to do next

8:45

is run you guys through high ROI ways to

8:47

combine these four sort of principles

8:48

behind system prompts and then apply

8:50

them both in global, local, and then

8:52

also give you guys sort of like a little

8:54

workflow loop that you can use in order

8:56

to understand how to update this

8:57

effectively. And this isn't just going

8:59

to be some big long system prompt that

9:00

I'm giving you guys like I think we

9:01

probably all seen floating around

9:02

various sources in the internet. The

9:04

reality is like Claude MDs are highly

9:05

personal devices.

9:07

But these are going to be a list of

9:09

short principles that will almost

9:10

certainly help you design better

9:11

projects and then get more done, whether

9:13

economically or otherwise. So, the way

9:15

that all this is organized within Claude

9:17

code is using two different scopes,

9:19

global and local. And if you didn't

9:21

already know, basically there are a

9:23

variety of different places that Claude

9:25

code upon initialization will look to to

9:28

get the prompts that is injected at the

9:30

very top of its context window.

9:32

Okay, the two big ones for us are the

9:35

user over here, which is equivalent to

9:37

your global, and then also the project

9:40

over here, which is equivalent to your

9:41

local. And so basically what this means

9:43

is if you have a file called Claude.md

9:46

all caps that exists within this folder

9:49

on your computer somewhere,

9:51

it'll load that up on every Claude code

9:53

session whether or not you're working in

9:54

the same workspace or another one.

9:56

Now, if you have a Claude.md capital

9:59

Claude. lowercase md located within a

10:01

dot Claude folder within your specific

10:04

repository directory, then it'll also be

10:06

loaded. And in this way, you know, you

10:07

sort of have like a global precedent

10:09

that's always injected at the top of

10:11

every single thing, okay? No matter

10:13

what. And then you also have sort of

10:14

smaller little um you know, local

10:16

Claude.md that's also injected. And

10:18

collectively when I say, you know,

10:20

system prompts from here on out, really

10:22

what I'm referring to is I'm referring

10:23

to both of these. I'm not just referring

10:24

to one of these.

10:25

And because global is injected on every

10:27

single run, there's sort of like

10:28

different strategies in order to divide

10:31

the four things that we just talked

10:32

about. Um basically on your global

10:35

Claude.md, makes more sense to put

10:36

high-level reasoning and then your own

10:38

personal beliefs. And then in local

10:40

Claude.md, makes more sense to insert

10:42

local low-level knowledge. So, stuff

10:44

like I just talked about with the um

10:46

workspace itself. So, you know, if I

10:49

were just enumerating all of these

10:50

things, up here, okay, you'd put your

10:53

preferences, like your global

10:54

preferences. These could be things like,

10:56

"Hey, you know, when you return a file,

10:58

I want you to return the absolute file

10:59

path to click on it because whatever

11:01

editor I'm using doesn't really have uh

11:03

take that into account." You know, it

11:04

could be things like uh programming

11:06

conventions. "Hey, I want you to program

11:08

using I don't know, object-oriented

11:09

programming." Or, "Hey, I want you to do

11:11

like functional programming in in Rust."

11:13

"Hey, when I ask you to develop a new

11:14

project, I always want it done in Rust

11:16

as opposed to, you know, Python." Or or

11:18

something like that. Alternatively, it

11:19

could be stuff like uh "Hey, you know,

11:20

if I ask you to do something using a

11:21

tool you're unfamiliar with, always go

11:23

and read the API documentation first

11:25

before attempting to start because every

11:27

other time that you've attempted to do

11:29

something without the API documentation,

11:30

you typically run into loops, you waste

11:32

XYZ tokens, so make sure to load the API

11:34

docs. By the way, if you can't load the

11:35

API docs through uh you know HTML, then

11:38

make sure to like load up a a Chrome

11:40

DevTools MCP server to go and get that

11:42

stuff, even if it's dynamically loaded

11:44

through JavaScript. Okay, so these are

11:46

high-level reasoning strategies. These

11:48

are your own preferences. These are your

11:50

own conventions. And then also um these

11:52

are going to be just sort of like agency

11:54

capabilities, so stuff like, "Hey

11:56

Claude, you can actually do X, Y, and

11:57

Z." If you believe that you can't for

11:59

whatever reason and you're wrong, you

12:00

can absolutely, you know, go and do

12:01

whatever you want. The local low-level

12:04

knowledge, okay, this is going to be

12:05

stuff like backslash init, which I'll

12:06

show you guys in a second. Um so it's

12:08

going to be like a compressed version

12:10

of all of the knowledge on your

12:12

workspace. Instead of Claude having to

12:14

in the future go through every single

12:16

file, it'll just be able to read the

12:17

Claude on the data sort of a loose

12:18

understanding, like, "Okay, what's

12:19

where? Why have we built this? What's

12:21

the purpose of this workspace?" and so

12:22

on and so forth. Some additional things

12:23

you can do are things like context about

12:25

you and your goals and your own

12:27

reasoning strategies, your own

12:28

communication styles. So I'm going to

12:30

give you guys examples of my own Claude

12:32

on the data in a moment where you guys

12:33

see that I actually give it a lot of

12:34

context about who I am and why I want

12:36

what I want. Um I'll run it through, you

12:38

know, reasoning strategies that I

12:39

personally use that have, you know,

12:40

yielded me a lot of success in the past.

12:42

They may actually not necessarily be the

12:44

optimal reasoning strategies, but which

12:45

I tend to understand and because I'm

12:47

communicating with this thing in every

12:48

freaking every 5 seconds nowadays, I'm

12:51

I'm better capable of understanding what

12:52

it's putting across if we use those

12:54

principles.

12:55

Um and then yeah, those high-level

12:56

preferences and then generally good

12:57

token conservation strategies. Whereas

12:58

with the local, you know, it's a

12:59

description of the project where

13:00

everything is low-level preferences,

13:01

like specific API docs and usage. If you

13:04

are using, you know, the GoHighLevel API

13:06

to do some project or whatever, you can

13:07

actually just like have the whole

13:09

GoHighLevel API existing within your

13:10

project. That'll minimize the number of

13:12

tool calls that um Claude has to make

13:14

to, you know, some sort of research

13:15

sub-agent go and do the thing for you.

13:17

Instead, it can remain local, reduce

13:19

total token usage, and then also just be

13:21

faster and then more accurate. And then

13:22

capabilities within the project. And

13:24

then that takes me to the local

13:25

workflow. So And then that takes me to

13:27

workflow. So, there's two sort of

13:29

workflows here that I want to talk

13:30

about. There's the local workflow, and

13:31

then there's the global workflow. And

13:32

the local workflow is going to be

13:34

responsible for updating our local

13:35

Claude and MD, and then the global

13:37

workflow is going to be responsible for

13:38

updating our global Claude and MD. Like,

13:40

it'd be nice if I could just give you on

13:41

a silver platter a bunch of stuff to put

13:43

in your Claude and MD, right? I think

13:44

that's what a lot of people want. But,

13:45

you're going to end up a much better

13:47

both developer and then a much more

13:48

productive person if you understand the

13:50

principles at play here and develop your

13:51

own.

13:52

So, initially to start, um anytime

13:54

you're developing anything in in Claude

13:56

code or whatnot, obviously you need to

13:57

plan your feature. And I say feature

13:59

here loosely, you know, I use Claude

14:00

code as basically like my business

14:01

assistant nowadays, and so I use it to

14:03

do anything from reading my emails to

14:05

grab me new summaries in the morning to

14:07

to communicate with XYZ people to design

14:09

me, you know, uh websites and so on and

14:11

so forth. So, feature here is really

14:12

loose. I'm not just talking like about a

14:13

five-coded project. I'm talking about

14:14

anything.

14:15

But, what you do is you start by

14:16

planning a feature, right? And then if

14:17

you think about it logically, what

14:18

Claude does next is it instantiates the

14:20

feature. However, over the course of

14:23

planning and instantiating, okay, it

14:26

will fail a bunch.

14:28

It'll also succeed a bunch of other

14:30

times. And ultimately, there'll be a

14:32

giant list of learnings between, you

14:35

know, step one to step two. And so, what

14:37

you do after you instantiate is you

14:38

actually compile all those learnings,

14:40

okay, into some efficient um high

14:43

information density thing that doesn't

14:44

see a lot of tokens, then use that to

14:45

update the Claude and MD.

14:47

And so, this is your local workflow for

14:49

managing your system prompt. And you

14:50

basically just do this every time. You

14:52

plan something, it'll do a bunch of

14:53

failures on the way, then you'll

14:55

instantiate it, you'll take all those

14:56

learnings, update your Claude and MD.

14:58

That way, the next time you plan a

14:59

feature, it'll already have all the

15:01

benefits of the failures, plus, you

15:02

know, any additional things that it

15:03

learned along the way.

15:05

And so, the first time around this loop,

15:07

you know, it might take uh I don't know,

15:08

let's say X time to develop a feature.

15:10

The second time around this loop, you

15:12

know, maybe shaved

15:15

off 10% of the the the search space, and

15:17

it's a lot faster. The third time you

15:19

go, maybe it takes 0.8X. Okay? And so,

15:21

like, the time will just get faster and

15:22

faster and faster every time until

15:23

eventually you develop things um using

15:25

Claude in a similar way that you would

15:27

develop if you were not using Claude.

15:29

Now, here's where it differentiates

15:30

between the global workflow. What

15:32

happens is, you know, as you accumulate

15:33

a variety of failures, successes, and

15:35

learnings, and so on and so forth, your

15:37

current local Claude MD gets really,

15:38

really good. After all that's done, what

15:40

you do is, you know, after hundreds of

15:43

these runs, okay, you can either pull a

15:45

slash insights feature or you can run

15:47

that yourself. Let me show you guys how

15:48

to do.

15:49

What this will do is this will compile

15:51

not like at a local level, but at a

15:52

global level all of the things that

15:54

Claude attempts

15:56

uh pretty consistently and then

15:57

struggles with pretty consistently. You

15:59

know, it's like, "Oh, hey, I noticed

16:00

that not only on that one project, but

16:01

also on more or less every project,

16:03

Claude consistently goes down silly

16:05

rabbit holes it doesn't need to um and

16:06

then tries coming up with its own stuff

16:08

instead of just consulting the docs."

16:09

And so, you know, after this is done

16:11

three or four times, obviously there's a

16:12

trend, right? So, what you can do is you

16:13

can take that information and then you

16:14

can pump that in your global.

16:16

Um after that, what I'd recommend is is

16:18

I'd recommend you manually review

16:19

because Claude is an agent at the end of

16:21

the day, and the more AI steps you have,

16:24

the more you compound probabilities, and

16:26

the less likely that it becomes that

16:28

Claude itself is making like the right

16:29

call. You know, if like Claude is

16:31

independently 90% successful on a task,

16:33

and then you give it to another Claude

16:35

which is 90% successful to a task, and

16:37

then you give it to another Claude, you

16:38

know, what you're really doing

16:39

mathematically is you're going um 0.9

16:41

raised to the three. And if you just do

16:42

a little bit of math there, that's not

16:44

90%, right? 0.9 to the three is 73%.

16:47

And so, I guess what I'm trying to say

16:48

is um the more steps you have without a

16:50

human in the loop here, uh the lower the

16:52

likelihood that your total determination

16:54

will be correct. And because this is

16:56

your Claude.md, it is your global

16:58

preference and convention file, it will

17:00

be applied to every future project.

17:01

Meaning, if there is a place you should

17:03

spend human time on, it is this exact

17:04

step here. So, I'd recommend manually

17:06

reviewing that. Once you manually review

17:07

that, then you can add some high high

17:08

ROI bullet points to your Claude.md and

17:10

so on and so forth, you know, just like

17:11

a high information density version, and

17:13

then you can actually update the the

17:14

Claude.md. And And you can repeat this

17:16

loop a few times if you'd like before

17:17

finally going back to the local loop.

17:19

And so I mean it's kind of like I don't

17:21

know what you want to call it, like an

17:22

infinity sign, okay? Kind of starting

17:23

here, you're going

17:26

kind of like this, and then you're kind

17:27

of looping back, and then you're just

17:28

doing this over and over and over and

17:29

over and over again. Obviously, you're

17:31

going to spend a lot more time in this

17:32

loop, but eventually you're going to go

17:33

down to this loop.

17:35

And this is how I personally develop

17:36

using Claude.in.md. This is why my

17:37

workspaces are super tight, and then

17:39

instead of me, you know, using a vanilla

17:41

version asking it, "Hey, go do X, Y, and

17:43

Z," and then it like stumbles around,

17:44

uses 20,000 of my tokens, and god knows

17:46

how many of my dollars. And when I say,

17:47

"Hey, I'd like you to do X, Y, and Z

17:49

thing. I'd like you to scrape some

17:49

leisure over." It already has all that

17:51

stuff baked in, while still being

17:52

flexible enough that I could change them

17:53

anytime that I want. Okay, so next, I'm

17:54

going to show you guys basically my

17:56

workflow every time I start with a new

17:57

project, assuming that I've already done

17:59

a little bit of work in the project. I

18:00

don't have a Claude.in.md, and I don't

18:01

really have any of that like advanced

18:03

tooling or system prompt harnesses stuff

18:05

set up. This is exactly what I would do

18:06

step by step. So, first of all, you need

18:07

to open up a folder. I was just lurking

18:09

about to my toes earlier. That is sort

18:11

of embarrassing.

18:13

But anyway, in antigravity, just go open

18:14

recent, and then I'm just going to open

18:15

up something. Why don't I do you know,

18:17

antigravity example right over here.

18:19

And you know, when I'm in this folder

18:21

right over here, obviously there are a

18:22

bunch of different files and, you know,

18:24

configurations. This one is using Gemini

18:25

for a while.

18:27

So, what I'd like to do next, I'd like

18:28

to open up a Claude code. And so, I'll

18:30

click on that button over here. Let's

18:31

close out the agent window cuz I'm team

18:33

Claude at least for the moment. Thank

18:34

you, space invader.

18:36

And really like the first thing that you

18:36

do is, you know,

18:38

you develop on your own. I always

18:40

recommend just like don't try baking in

18:41

any opinions into a Claude.in.md until

18:43

you've at least developed with out a

18:45

Claude.in.md or some sort of advanced

18:47

system prompt for a little bit. And the

18:48

reason why is because like you'll find

18:49

Claude's actually really good out of the

18:50

box. As mentioned, they're incorporating

18:52

more and more of these features natively

18:54

within it.

18:55

And so, like it's it's great. It's not

18:56

like the harness that makes the

18:57

intelligence. It's obviously the

18:58

intelligence inside of it that sort of,

19:00

you know, communicates with your system

19:01

prompt to to get good. But right now,

19:03

it's already fantastic. Anyway, after

19:05

you've done some developing for a while,

19:06

this is obviously some sort of website

19:07

here. It's like a template using Veet.

19:09

Just go {slash} init, just like that.

19:11

And basically, {slash} init will go

19:12

through, read every single file in your

19:14

workspace, which I'm currently doing

19:16

with fast mode, if you're wondering why

19:17

this is probably faster than than what

19:19

you're doing.

19:20

And then at the end of it, it'll come up

19:21

with a basically like a highly optimized

19:22

Claude.md that succinctly and

19:26

effectively summarizes the placement of

19:28

everything here. And you can see it just

19:30

generated one called Claude.md. So,

19:32

comes with the build dev lint commands.

19:34

Note that no test framework exists from

19:35

architecture review key dependencies and

19:37

their roles, then some style conventions

19:39

as well. So, now I'm going to open up

19:41

this Claude.md. Okay, and why don't we

19:43

just move this over to the main window,

19:44

so it's a little bit easier to see. And

19:46

you can see that more or less it it just

19:47

at a very high-level summary takes every

19:50

single line in my entire workspace, then

19:53

it just significantly increases the

19:54

information density at a cost of like

19:56

total comprehensiveness. So, what I have

19:58

now is I is I have a summary of

19:59

everything.

20:00

So, that means is the next time that I

20:02

ask Claude anything about my workspace,

20:04

okay, the next the next go around,

20:06

um I don't actually have to like have it

20:08

like run through every single thing in

20:10

the file. Like, for instance, what I'm

20:11

going to do here is I'm just going to

20:13

call this like

20:14

um I don't know, xyz.md.

20:17

Or actually, you know, why don't I just

20:18

delete this for now.

20:20

You know, if I had asked this Claude

20:21

version something

20:23

about dark mode, "Hey, what are my

20:26

opinions on dark mode?" It's going to

20:28

check its memory for notes on the

20:29

preferences, not going to find anything.

20:30

And notice how it's just going to say

20:31

there's there's there's nothing at all.

20:33

So, what I could say is read through

20:35

whole project and find my preferences.

20:38

And now what it'll do is it'll, you

20:40

know, essentially launch some sort of a

20:41

genetic search with readmes and so on

20:42

and so forth until it finds something

20:44

about dark mode. In this case, it was in

20:45

the Gemini.md.

20:47

Um but I want you guys to know that, you

20:48

know, whether or not you have it in a

20:49

Gemini.md or it's just sort of written,

20:51

it'll eventually figure it out.

20:53

Now, the issue is, you know, how what

20:55

what sort of usage did we just uh do in

20:57

order to get that?

20:59

If I just scroll all the way up here, uh

21:00

type {slash} contexts, you know, the

21:02

system prompt was 0.6%. Free space was

21:05

messages was 0.9%. So, that last message

21:07

chain there with the tool calls and

21:09

everything like that might have

21:09

realistically taken like 5 or 6,000

21:11

tokens. I don't need to do that sort of

21:13

thing ever again. You know, if I bring

21:15

that back and go cloud.md and then if I

21:17

just open up a new instance and I say,

21:19

"Hey, you know, what are my opinions on

21:21

dark mode?"

21:22

Obviously, it's going to read the

21:23

cloud.md and you know, instead of me

21:25

having to use god knows how many tokens.

21:27

If I go back to {slash} context, you

21:29

know, you see that I've now you only use

21:30

0.2%. So, basically save myself what's

21:32

that like 6,000? And let me tell you

21:34

these cloud tokens ain't free, man.

21:35

Anthropic's increasing the price uh

21:37

pretty aggressively especially recently

21:38

when they realized 99% of the world is

21:40

now operating using their infra. So, I

21:42

guess what I'm trying to say is I'm

21:42

spending like literal like money, but

21:44

I'm also spending time and to me the

21:45

bigger thing is time. But, what are some

21:47

other things asked? I mean, like think

21:48

about deployment. If you have any sort

21:50

of like front-end or full-stack

21:51

experience, you'll know like, you know,

21:53

usually the flow is you start with a dev

21:54

server. You use that dev server via npm

21:57

run dev or equivalent to like figure

21:58

things out on your uh you know, develop

22:00

various features and so on and so forth.

22:02

Then you'll build, you'll do some sort

22:03

of linting and then once you're done,

22:04

you'll have to like preview it, you'll

22:05

you'll push to production or or sorry,

22:07

you'll push to staging and then verify

22:09

that and then eventually you push to

22:10

production, right? Like obviously, this

22:11

is something that I could have learned

22:12

just by going through the folder

22:14

structure, seeing source, public, node

22:16

modules, all these things. But, you

22:18

know, I'm just listing them out over

22:19

here so that instead of you having to

22:20

actually read any of that filing or

22:22

tooling, you know, you can do it in god

22:23

knows how many what's that? Five tokens,

22:25

six tokens or something immediately.

22:27

Likewise, you know, I see where things

22:29

are laid out. So, in this case, this is

22:30

obviously a single page application. The

22:32

entire app lives in a single component.

22:33

Nav here or services project sections

22:35

all markup and logic is here.

22:37

Um it is evident if you were to actually

22:39

click on app.jsx and then scroll through

22:41

that that is the case. But, look at how

22:43

many more tokens app.jsx is versus, you

22:45

know, just that brief little description

22:46

in uh cloud.md. If I were to copy and

22:49

paste the entirety of this into

22:50

something like a word counter, you could

22:51

see it's 827 words approximately 1,100

22:53

tokens. Okay, if I go back to my

22:54

cloud.md, like how long is how long is

22:56

this? It's 22. So, that's a what? 45x

23:00

compression ratio. That sort of

23:02

compression is how you ultimately get a

23:04

significantly better and more effective

23:05

clot because you are not shoving a

23:07

tremendous amount of tokens at the

23:08

beginning of any query.

23:10

Um and you know, as we hopefully know,

23:12

uh token length tends to scale inversely

23:14

with the quality of the output. The more

23:16

tokens in a context window, not only the

23:18

more money are you spending, but

23:19

typically the lower quality of the

23:21

results are. So, just avoid all that by

23:22

initializing and then storing a bunch of

23:23

information about, you know, what the

23:25

project is on. Uh you'll be you'll be

23:26

much happier for that. But, you know,

23:28

slash init isn't the only thing that I

23:29

would do. From here, I'd actually start

23:31

importing a couple of my preferences and

23:32

then things that I've tried. So, I don't

23:34

know, let's just say I'm going to remove

23:35

the gemini.md for simplicity. Let's just

23:38

say I'm developing a new feature and um

23:40

actually, why don't we just visualize

23:41

app? I don't know okay, what it looks

23:43

like. Let me actually take a look at

23:44

this thing. So, it'll run the dev server

23:46

so I can see in the browser. And

23:46

immediately I'm thinking like, "Hey, you

23:48

know, this is actually kind of

23:48

inefficient." When I say visualize app,

23:50

I basically just want you to launch it.

23:52

So, store in your cloud.md that when I

23:54

ask you to run the dev server or open

23:55

the app, I just want you to open it in

23:57

my Chrome instance as well. I don't just

23:58

want you to run the dev server. You

23:59

know, basically, next time I say this, I

24:01

don't want it just to like say, "Hey,

24:02

the dev server's here, give it a click."

24:03

And then I'm like, "Okay, can you just

24:04

open it cuz I'm already here?" Um I just

24:06

want it to open it automatically, right?

24:08

Okay, cool. And I see, you know, it's

24:09

kind of over here. Um so, that's nice.

24:11

Definitely not a fan of the design. I

24:12

don't like how it scrolls through. I'll

24:13

just say, "Hey, I want you to

24:14

significantly improve the perceived

24:15

visual quality of the application. Go

24:17

and look up uh you know, the Apple

24:19

website and then compare that to our

24:21

website. Make some changes that just

24:22

improves both the perceived visual

24:24

cohesiveness, quality, etc." Must have

24:26

been a Gemini website. Anyway, let's uh

24:28

see what it does here. It's fetching

24:29

Apple site for design reference, reading

24:31

the current app code in parallel. And

24:32

now it's just doing a bunch of updates,

24:33

editing the CSS, nav-link,

24:35

hero-background, and so on and so forth.

24:37

And you can see that it is actually

24:38

updating the site. I mean, it's doing it

24:39

currently in real time, but uh it's

24:41

looking significantly better. It's also

24:42

picked up some new colors and so on and

24:44

so forth. Now, what's cool is it

24:45

actually just opened up a second uh

24:47

project for me right over here um

24:49

because earlier on I stored my

24:51

preferences that I don't want it to just

24:52

give me the link. I actually want it to,

24:54

you know, open open stuff up. So, that's

24:56

nice. Obviously, we have better

24:57

designing and stuff like that. But, the

24:58

key part here is when I say, "Okay,

25:00

great. Nice job. How could you have

25:01

arrived at these conclusions and done

25:03

everything that I just asked you to do

25:03

faster?" Okay, and now look, we're

25:05

already at the point where we're capable

25:06

of optimizing a fair amount of these

25:07

design issues. Instead of 20 edit calls,

25:10

which is what I did before, what I could

25:12

have done is just do one right call. So,

25:14

the reason why that took like 30 seconds

25:15

or so because it was editing the CSS

25:17

file line by line across 20 sequential

25:19

tool calls. I should have read the file,

25:20

rewritten the whole thing in my head,

25:21

done a single right to replace index.css

25:23

in one shot.

25:25

Yes, I'd like you to save this in the

25:26

cloud.md. Do it as a user preference

25:28

section. So, asking it questions like,

25:30

"How could you have arrived at those

25:31

conclusions and done everything I just

25:33

asked you to do faster and for fewer

25:34

tokens?" is pretty powerful. Doing this

25:36

consistently as you develop and design a

25:38

project and then having a running log of

25:40

changes to the cloud.md is also quite

25:41

valuable.

25:42

Another thing you can do is you could

25:43

set a meta prompt in the cloud.md and

25:45

that's personally what I always do. That

25:46

basically says like, "When you have made

25:48

a mistake, I want you to update the

25:49

cloud.md with a running log of things

25:51

not to do next time.

25:53

When you've made a mistake, I want you

25:54

to update the cloud.md with a running

25:56

log of things not to try next time."

25:59

Essentially, I want this to be almost

26:00

like a mini experimenter's or research

26:03

person's notes that shows what a future

26:05

cloud instance should not do while

26:08

working on this project. Update the

26:10

cloud.md to reflect what I just said at

26:11

the very bottom. Okay, now it has a

26:13

section called lab notes what not to do.

26:14

This is going to show a bunch of

26:16

failures as well as learnings and

26:18

successes and so on and so forth. And

26:19

we're already honestly like halfway down

26:21

the loop. Now, this is a very contrived

26:23

example because I'm literally just

26:24

building a website. But, imagine that,

26:25

you know, instead of just a website,

26:27

you're building a workspace that is

26:29

meant to contain all of your business

26:30

basically entirely. All of your SOPs

26:32

it's meant to contain all of the work

26:34

that you do on a daily basis. It's meant

26:35

to contain your to-dos and so on and so

26:37

forth. Having information like what I

26:38

just showed you for this project would

26:40

be invaluable across more or less all

26:41

levels of both development and then also

26:44

productivity. And that's personally what

26:45

you should ultimately be working

26:46

towards.

26:47

So, anyway, we can make this as

26:48

complicated as we want, obviously, but

26:50

hopefully you guys see that loop at

26:51

work. We plan a feature, so we just did

26:54

this. It was simple enough that we

26:55

didn't need to use a dedicated plan

26:56

mode, but obviously I still one-shotted

26:57

it. After it implemented the feature,

27:00

along the way it did a few things that

27:01

realistically could have done better.

27:03

So, what do we do after? We take those

27:04

learnings, we compile them, and then we

27:06

update the Claude and MD. And this is

27:07

sort of a meta example since I literally

27:09

was doing it while I was building the

27:10

Claude and MD, but hopefully you guys at

27:12

least understand conceptually I'm sort

27:14

of what you do. After four or five of

27:15

these runs, there's probably a fair

27:17

amount of stuff here that you can take

27:19

advantage of. And that's where an

27:21

insight run would make sense. So, let me

27:23

actually zoom in and then just delete

27:24

this so you guys could see.

27:26

In case you didn't know, insights is a

27:27

simple slash command that basically runs

27:29

a bunch of sub agents across all of your

27:31

Claude conversation history. The benefit

27:33

to that is now not only are we running,

27:35

you know, and and changing our local

27:36

Claude and MD,

27:38

we're also evaluating all of like the

27:39

patterns in communication that we've had

27:41

with Claude and MD over the course of

27:42

the last, I don't know, could have been

27:43

like few days, could have been months,

27:45

could have been, I mean, years depending

27:46

on how soon or late rather you are

27:48

watching this video. So, um just like we

27:50

optimized our local Claude and MD, now

27:52

we can start optimizing our global. And

27:53

while it's chewing away because insights

27:55

does take a fair amount of time, okay,

27:57

I'm just going to create a new file

27:58

here.

27:59

I'm going to call it global Claude.md.

28:01

I'm just going to give you what I would

28:03

consider to be, at least as of the time

28:04

of this recording, probably like some of

28:05

the higher ROI principles to make sure

28:08

to include. Um I include this in my own

28:10

global Claude and MD because I think

28:12

it's just very very valuable. So, I'll

28:13

say global Claude.md.

28:16

This is inserted at the beginning of any

28:17

conversation with Claude across all of

28:20

the user's workspaces. So, first I'd

28:21

have a profile section, so this is a bit

28:24

about Nick.

28:26

So, you know, I don't know, it'd be like

28:28

Nick is a 30-year-old

28:30

and J

28:33

high-performing

28:35

internet entre

28:36

preneur.

28:38

He runs a YouTube channel at 350

28:42

subscribers or 350 by the time I make

28:43

this uh publish this video. 350,000

28:45

subs, an Instagram channel,

28:48

and so on and so on and so forth.

28:50

Okay, and so I have a bunch more

28:51

information which I've taken from just a

28:53

couple of other systems I built. Um this

28:56

one here is Nick is a 30-year-old and

28:58

Jay, here's his revenue. So, here are

28:59

all the different things that contribute

29:01

to my revenue. Here's some churn math,

29:03

um some of the companies that I'm

29:05

currently owning, uh some teams, right?

29:07

So, it's me, it's an editor, it's a

29:09

LinkedIn newsletter person, it's a bunch

29:10

of AI agents, bunch of information on

29:12

YouTube as well as my goals, and then

29:14

ultimately I'm on Instagram as well. And

29:16

you're thinking like, Nick, this is

29:17

crazy. Why would you insert all this

29:18

information in your global Claude and

29:19

MD? Well, the reason why is because I

29:21

want this on every conversation that I

29:23

have with it to understand who I am and

29:25

to take that into consideration when

29:26

discussing things with me. I can't say

29:28

how many times I'm having a conversation

29:30

with Claude and because I don't have

29:32

context like this cuz I'm in a naive

29:34

thing with no personal system prompt

29:35

injecting the context when I know I say

29:37

something along the lines of, "Hey,

29:38

what's the best solution for X, Y, and

29:39

Z?" And then it says, "Oh, you're going

29:41

to want to do this solution." And then I

29:42

say, "Why?" And then it'll say, "Oh,

29:44

because it's the cheapest." Right? It

29:45

only costs 0.2 cents, whereas the all

29:46

other solutions cost $5. And I'm

29:49

thinking, "Well, if you knew a little

29:50

bit about who I am, you'll know that

29:51

money is not the primary bottleneck

29:53

right now. I prefer you to exchange my

29:54

my money for my time."

29:56

Um so, just giving it some like

29:57

high-level principles like that is is

29:58

very important. Anyway, while I was

30:00

doing that, um the actual Sherpa AI

30:02

Insights report um is ready. So, I'm

30:04

just going to tell it to open it so I

30:05

can take a look at it with you guys.

30:07

And now you'll see there's an HTML page

30:09

basically that runs through everything

30:10

about Claude, all of the insights across

30:13

all of the sessions. Looks like 1,000 AI

30:15

or 49 messages across 200 sessions. I

30:17

don't know where this chooses the cut

30:19

off. It looks like it's like about a

30:20

month or so.

30:21

Um although keep in mind that like this

30:23

is Claude code specific and I don't know

30:25

if this encapsulates all the

30:26

conversations I've had with it on the

30:27

desktop app, but pretty good. And you

30:29

can see here that, you know, there's a

30:30

bunch of context about what I work on

30:31

and and so on and so on and so forth and

30:33

how I use it and all this stuff.

30:35

So, the the important thing to do is

30:38

existing features to try section. You

30:40

can just copy this in the Claude code

30:41

and add it in Claude in MD. So, for

30:43

instance, when using Chrome DevTools MCP

30:45

or browser automation, always kill stale

30:46

Chrome processes in a clean profile

30:48

before starting. If the MCP tools fail

30:50

twice, stop and ask the user for

30:51

continuing to retry. Never waste tokens

30:53

on repeated failing browser calls. This

30:55

is actually quite valuable just given

30:56

how many times I have tried to have it

30:58

run, you know, Chrome DevTools MCP and

31:00

it's failed.

31:01

Um, same thing over here.

31:03

Same thing over here, you know, with

31:04

some face swap information and stuff

31:06

like that. You can copy all this in the

31:07

Claude and it will set it all up for

31:09

you, which is pretty valuable. As well

31:10

as it it can even go and build like new

31:12

skills based off of things that you

31:14

consistently ask. So, that's that's more

31:15

also what I'm doing here.

31:17

Um, anyway, the the value with this is

31:18

basically to like copy the entire thing,

31:20

go back here, paste it in.

31:23

Can say, "This is my Claude insights

31:25

file. It describes at a high level a few

31:27

of the obvious design patterns in my

31:28

thinking, and then a couple of the

31:30

issues that I've had communicating with

31:31

you and other versions of you. I'd like

31:33

you to distill this into a list of high

31:35

information density snippets that I can

31:37

paste into a global Claude.md to be both

31:39

token conservative, but also avoid most

31:41

of the mistakes that you typically

31:42

make." And I'll just press enter. It's

31:44

going to give me some information about

31:45

that. And over here, we actually have

31:46

the changes. And this is very high

31:48

information density, right? It basically

31:49

took a bunch and said, "Don't

31:50

overexplain, overengineer, add un-

31:52

requested improvements. When making

31:54

widespread changes to a file, use one

31:56

write instead of many sequential edit

31:57

calls. Speed matters. Don't fetch

31:59

well-known websites." Again, a rerun

32:01

browser automation and, you know, so on

32:03

and so forth. Just Just some high-level

32:04

stuff.

32:05

It looks like it just inserted that in

32:07

here, which is quite nice. So, now, what

32:09

do we have? We have, if you remember,

32:11

some context on me in the global

32:13

Claude.md. We also have some high-level

32:15

reasoning rules and principles. And

32:17

really, what we're just missing is some

32:19

um token conservation strategies. And

32:21

you could see this by uh you know, you

32:23

can go back

32:24

uh rewind the video if you'd like some

32:25

more on that. But, basically, you want

32:27

context about you, your goals, and your

32:28

reasoning strategies, some high-level

32:29

preferences about, you know, what it is

32:31

that uh it is currently doing that is

32:33

wrong that you would like it to fix, and

32:34

then some good token conservation

32:35

strategies like not docs first. So, what

32:37

I'm going to do is underneath

32:39

interaction rules, I'll also just say

32:41

oh, and what's really interesting I'm

32:42

seeing one of my rules are actually

32:44

directly contradicting some of the other

32:46

rules. Don't fetch well-known sites.

32:47

I'll actually just remove that.

32:49

Um that's the human in the loop part,

32:50

right? Just look to see if any two rules

32:52

directly contradict each other. Then

32:54

I'll say

32:55

when a user asks you to use a

32:57

non-trivial platform,

32:59

one for which you do not have contacts

33:01

in, always look up the documentation

33:03

first. You can do so by looking into API

33:06

documentation plus platform name.

33:09

After, if for whatever reason you can't

33:10

access the docs for JavaScript reasons,

33:13

launch a Chrome DevTools MCP Chrome

33:15

instance so that you could still copy

33:16

and paste all that data. No matter what,

33:18

if you're working on a project for whom

33:20

API documentation is available, you

33:21

should always go through the API

33:23

documentation to avoid 99% of the

33:25

errors. The tokens we spend reading the

33:26

docs will save us a lot of tokens in

33:28

trying to use things that don't work.

33:29

Cool. So, I'm going to copy that.

33:32

And now I have my global Claude in MD.

33:34

And you know, I could always just have

33:35

Claude actually insert that into the

33:36

global Claude in MD. Um I could also

33:38

just like go and find the find the

33:39

finder. So, I'm going to go to finder on

33:41

Mac. Basically, you can find your global

33:43

Claude in MD just by going to your Mac,

33:45

in my case users, I'm on next to arrive.

33:47

And then um there's a hidden folder here

33:49

which you can't actually see just right

33:51

out of the gate. You should be able to

33:52

go uh shift command I think it's comma

33:56

or period. There you go. Uh shift

33:57

command period. Once you're done with

33:59

that, you can scroll all the way down to

34:00

where it says Claude. And then over

34:01

here, um you'll see that there is a

34:04

Claude in MD that lives within that

34:05

Claude. So, what I can do now is I can

34:07

just reveal this folder in my finder,

34:09

compare it to that folder in my finder,

34:11

and I can actually just go drag and drop

34:13

this in.

34:14

Now I have global Claude, and I can just

34:16

remove this Claude.

34:17

And replace that with this Claude.

34:19

Awesome. So, now all future

34:20

conversations that I have with Claude

34:21

across all of my uh workspaces and all

34:24

of my folders will include the

34:25

information that I just provided. And

34:26

hopefully you guys see how simple it is

34:28

to run that loop. Um granted, this is an

34:29

informal loop. I'm not really showing

34:31

you guys like a simple formal streamline

34:32

process, but hopefully you see how easy

34:34

it would be to build that in again as

34:36

like a meta Claude.

34:38

Let's talk a little bit about agent

34:40

harnesses. So, agent harnesses, the term

34:43

anyways, gotten a ton of interest over

34:44

the last couple of months because it's

34:46

sort of new and exciting, but very few

34:49

people actually understand what it

34:50

refers to and what it means.

34:52

An agent harness, to be clear, is just

34:53

Claude code. Claude code is the harness

34:57

around the model Claude that enables it

35:00

to do things like call various tools and

35:03

get actual economically valuable work

35:05

done.

35:06

For those of you that don't know, um all

35:08

that like AI models are are just text

35:11

interfaces, right? It's just text in

35:13

text out. A harness is what turns

35:15

something that can only communicate in

35:17

text into something that is ultimately

35:18

capable of like controlling our

35:20

computer.

35:21

So, the way that I personally think

35:22

about the question, "What is a harness?"

35:25

is a harness is just everything that

35:27

wraps around the LLM that is not the

35:29

actual LLM itself.

35:31

So, in our case, it's Claude code. It's

35:34

the system prompt. It's the hooks. It's

35:36

the tools that it has access to, and

35:38

it's the parameters therein that control

35:40

things like when the memory auto

35:42

compacts, how many messages you can send

35:45

in a turn, what the total number of

35:47

token limits are, and so on and so

35:49

forth. For the purposes of this demo,

35:51

let's pretend that this over here is our

35:52

Claude Space Invader, okay? And so, this

35:55

is sort of like the the the large

35:57

language model itself. This is actual

35:59

Claude.

36:01

And so, Claude is obviously like a

36:02

galaxy brain intelligence. It's been

36:04

trained on god knows how many books and

36:06

blog posts and encyclopedias and so on

36:08

and so forth. But, you know, Claude is

36:11

sort of it sort of exists in this

36:13

boundary where it can't actually do

36:14

anything outside the real world unless

36:17

it's given the tools and the ability to

36:19

do so.

36:20

And so, um one example of things that

36:24

Claude has access to are said tools. So,

36:27

that's things like uh I don't know, the

36:29

ability to use bash, okay, use a

36:30

terminal, the ability to use I don't

36:32

know, grep, which is how it finds things

36:34

around your computer, and so on, and so

36:36

on, and so forth.

36:37

Another thing that it has access to,

36:39

kind of going back and forth, is some

36:40

form of memory ink, right? What it can

36:43

do is it can read, so it can read things

36:45

that are stored in this memory, and then

36:47

it can also write, so it can add, sort

36:48

of update things as needed.

36:51

You know, there's obviously also a

36:52

variety of other things here that it has

36:54

access to. And you know, if it didn't

36:55

have access to all these things, again,

36:56

it would just be like an agent or a

36:58

model, sorry, that exists in the box.

37:00

And so, that's really the difference

37:02

between, you know, LLMs and agents.

37:05

Agents are LLMs plus a harness, whereas

37:09

LLMs by themselves, large language

37:11

models, they can't really do anything.

37:12

They obviously um

37:13

operate entirely in the domain of

37:15

knowledge. So, just given the fact that

37:17

it's called a harness, you can kind of

37:18

think of it as, you know, I'm going to

37:20

draw a really crappy dog here. Put

37:22

another way, here's a really crappy

37:23

rendition of a what I initially wanted

37:25

to be Canadian dog sledding, and what

37:27

ended up being looks more like Santa

37:29

with a big fat beard riding a

37:32

questionable reindeer. Uh but basically,

37:34

you can imagine that like this right

37:36

over here, this is your LLM, this is the

37:38

actual model intelligence, and then you

37:40

over here, okay, this is your harness.

37:42

This is actually like the the code part

37:44

of Claude code that sort of controls it.

37:46

And so, the LLM wants to go in a bunch

37:48

of different ways, it wants to do a

37:49

bunch of things. What the harness does

37:50

is it just sort of narrows down its

37:52

direction. And you know, you can kind of

37:54

almost think of it like uh the barrel of

37:56

a gun or something like that, right?

37:58

Whereas, you know, back in the day, you

38:00

might have had like cannons, and then

38:02

you might have loaded those cannons with

38:04

big uh

38:06

massive cannonballs, and they're huge.

38:08

And what you'd do is you'd stuff some

38:09

additional gunpowder underneath and

38:11

stuff like that. And those cannons would

38:12

kind of, you know, despite the fact that

38:14

they were operating off the same

38:15

fundamental technology, which is

38:16

gunpowder, they might not really be able

38:18

to go so far. I don't know, let's just

38:19

say 50 m.

38:21

Nowadays, obviously, we have um this is

38:23

my really crappy gun drawing. With, you

38:25

know, more or less the exact same

38:27

technology, you put some sort of bullet

38:29

in there, right? But then, because of

38:31

the technology that surrounds the core

38:34

thing, which is the gunpowder, you know,

38:36

the bullet can go a lot farther. So,

38:38

maybe instead of 50 m, now it can go, I

38:39

don't know, 250 m or so.

38:42

So, this is how I this is how I think

38:43

about harnesses, okay? And I don't mean

38:44

to just show you a bunch of silly grade

38:45

school analogies, but it is important to

38:47

realize that like that is what um now

38:49

Claude Code really is. And because

38:51

Claude Code is a harness, obviously,

38:52

there are a bunch of other people that

38:53

have tried making their own harnesses as

38:55

well. Just like we have frameworks like

38:57

React and View and then Next.js and and

39:00

and Knox, we also have a bunch of

39:01

different harnesses that have been

39:02

developed that supposedly work on and

39:04

then improve on on specific aspects.

39:06

What are some of those aspects? Things

39:07

like security, right? Automatic

39:09

permissions, so plan mode versus default

39:11

mode versus the new enable auto mode and

39:14

then bypass permissions mode. You know,

39:16

there are some harnesses out there,

39:18

okay? There are some a gigantic SDKs and

39:21

stuff like that, not going to name any

39:22

names, but

39:23

um there are some of them that are

39:24

probably a little bit less secure than

39:25

others. Such that if they were to read a

39:28

Twitter thread that looked like this,

39:29

maybe they would actually execute sudo

39:31

rm -rf and delete your entire hard

39:33

drive, right? Bunch of examples of

39:35

people screwing around with this. This

39:36

is an example of Codex, which um you

39:38

know, being an extraordinarily competent

39:40

model, I can't really talk down too much

39:42

on. But this is an actual conversation

39:43

that uh you know, it had with somebody

39:45

that I found on Twitter. You know, the

39:47

model basically tried to running

39:48

something that was like rm -rf, which to

39:50

make a long story short, in case you

39:51

didn't know, just deletes everything.

39:53

And uh here it says, "Well, the shell

39:54

policy actually blocked the raw rmrf.

39:56

So, what I'm doing is I'm removing those

39:58

generated directories like in the shell

40:00

policy with a Python cleanup instead.

40:02

Same effect, less policy friction,

40:03

right?" It's just going to go ahead and

40:04

delete the entire thing. You know, the

40:06

the harness impacts a model's ability to

40:08

get things done. It also impacts

40:10

ultimately the safety. It impacts like

40:12

the memory and so on and so on and so

40:14

forth. And so, hopefully, at least now

40:15

you guys understand what the harness is

40:17

before I show you guys some examples of

40:18

different versions of it. Obviously

40:20

Claude code is the major harness today,

40:23

but there's a great blog post over here

40:24

by LangChain that more or less describes

40:26

a way to create different harnesses.

40:29

The model gets a certain type of context

40:31

injected into it prompts memory skills

40:33

or conversation. Then you also have

40:35

orchestration, things like Ralph loops

40:37

which is really big a while back that

40:38

was a different type of harness.

40:40

You know, there's a certain persistence

40:41

of data, actions, and then the ability

40:43

to both observe and verify so with

40:45

screenshots and stuff like that.

40:47

One harness that a lot of people are

40:49

using now is this sort of droid idea,

40:52

which was built by Factory AI. So droid

40:54

is like a publicly available harness

40:56

that you can run and download today.

40:58

pi.dev is also exploding in terms of

41:00

popularity. So whereas Claude code, you

41:02

know, obviously needs to run with Claude

41:04

infrastructure, right? Claude is the

41:05

model underlying Claude code. This pi

41:07

coding agent is sort of like the

41:08

open-source provider of it. You can feed

41:10

in more or less anything that you want

41:12

including Claude and then just have it

41:13

operate inside of this this harness. And

41:16

you know, what this does is it just

41:17

changes the way that we store memories,

41:19

it changes the way that we store certain

41:20

files. It's sort of like modifies, it's

41:22

almost like an alien or bizarro version

41:24

of Claude code in so far that it changes

41:26

a few of like the fundamental constants

41:28

like how long before context compaction,

41:31

you know, how to try different types of

41:32

solutions and stuff like that. Various

41:34

baked-in behaviors regarding Claude code

41:36

and and so on and so forth. And the

41:37

reason I'm covering this is because, you

41:39

know, this is something that was very

41:40

fundamental to Anthropic. The back end

41:41

of November the 26th, 2025 they wrote a

41:43

big long blog post called Effective

41:45

Harnesses for Long Running Agents, which

41:47

at the time kind of changed the game.

41:48

And I would say this is the beginning of

41:49

the kickoff of Claude code superiority

41:51

over most other harnesses.

41:53

And so, you know, here it describes

41:55

various different ways to work on long

41:57

running coding projects and manage

41:58

environments and stuff like that. And so

42:00

obviously this is something that's like

42:01

very fundamentally baked into Claude

42:02

code. If you want to understand Claude

42:04

code at an advanced level,

42:05

you can't get better than getting it at

42:07

a harness level. Okay, so you know,

42:09

obviously this is a Claude code course,

42:10

it's not another harness course, but you

42:11

should at least know what agent

42:12

harnesses are before you proceed to the

42:14

rest of the course because, you know,

42:16

the more understanding of harnesses you

42:18

have, I think the better you'll be able

42:19

to appreciate and then I digest and

42:21

ultimately execute on what I'm about to

42:22

show you.

42:24

Next, I want to chat a little bit about

42:26

parallelization, about things like agent

42:28

teams, about sub-agents, and a couple of

42:30

other ways of distributing work to

42:32

minimize the amount of time and effort

42:34

that goes into things while also

42:35

increasing the quality of the output.

42:37

Okay, so I say agent teams here, but

42:39

let's start with parallelization. A big

42:41

question that I think a lot of people

42:42

have is, well, first of all, what the

42:43

heck is parallelization, which is just

42:45

doing multiple things simultaneously

42:46

instead of waiting for sequential things

42:48

to finish. And then the second one is

42:50

like, Nick, why the hell should we

42:51

parallelize our agents to begin with?

42:52

And to that I say, have you ever, you

42:54

know, sent a long-running task request

42:56

to Claude code and actually had Claude

42:57

execute on something for more than a few

42:58

minutes? For the vast majority of the

43:00

time, you're just sitting there

43:00

twiddling your thumbs. Twiddling your

43:02

thumbs is not very economically

43:03

productive, so if I have ways to not

43:05

twiddle my thumbs, I will do so. And I

43:06

really, I guess mean is that autonomous

43:09

agents just take a long time to finish

43:10

tasks.

43:11

You know, when we started with this

43:12

stuff, or at least when I started with

43:14

this stuff last year, you know, Claude

43:16

could realistically work on things for

43:17

30 seconds.

43:18

The other day, I had Claude work on

43:19

something for over 15 minutes.

43:22

And so, if all I'm doing is just sitting

43:23

there waiting for it to do this

43:24

15-minute task, you can imagine that my

43:26

productivity is basically going to be

43:27

punctuated by me just sitting around

43:29

watching it, it does something, I get

43:31

the result, make some minor changes,

43:33

wait for another 15 minutes, and so on

43:34

and so forth. That's not very efficient.

43:36

So, parallelization allows us to reduce

43:38

the total amount of time by a factor of

43:39

at least a few, from 15 minutes to maybe

43:42

a couple minutes, so it'll be able to

43:43

work on smaller, more more

43:44

self-contained things.

43:45

Um the two will also just get power

43:47

quality.

43:48

Another thing is that many tasks feature

43:49

independent steps that can be broken

43:51

down. So, for instance, let's say I'm

43:52

doing some sort of task, okay? And this

43:54

is just like how long it would normally

43:56

take if you go serially. And so, the

43:58

option A is just to do what most people

43:59

do, which is where they'll do, I don't

44:01

know, they'll do step one, and then

44:03

they'll do step two, and then they'll do

44:04

step three, and then they'll do step

44:06

four.

44:07

So, that's 1 2 3 4.

44:10

This task over here takes 5 minutes.

44:12

This task over here takes 5 minutes.

44:15

Stack over here takes 5 minutes. And

44:16

this task over here takes 5 minutes.

44:18

What's the total amount of time kind of

44:20

collectively? Well, it's 20 minutes,

44:22

right? So, that's sort of A you know,

44:24

the serial way that most other people do

44:25

things. Well, guess what? Turns out a

44:27

lot of tasks don't need to necessarily

44:29

be like that.

44:31

If I just copied all of this stuff over,

44:33

okay? And then instead ran a couple of

44:35

these in parallel, so I actually had I

44:37

don't know, three of these

44:38

simultaneously and then kind of combined

44:41

all of them. If I did something maybe

44:42

more akin to this instead. Hopefully you

44:45

guys could see now, okay? Instead of

44:48

everything taking

44:49

you know, 5 minutes, 5 minutes, 5

44:51

minutes, and 5 minutes. Maybe what I'm

44:52

capable of doing realistically is this

44:54

takes 5 minutes.

44:55

This takes 5 minutes. And then the

44:57

integration step about these three which

44:59

were two, three, and four

45:02

only takes 2 minutes. So, what I'm doing

45:04

is I'm basically converting a task that

45:05

previously took 20 minutes and I'm

45:07

turning it into one that took 12

45:08

minutes.

45:09

Which you know, if you just did a little

45:10

ratio, 12 over 20 is equal to 3 over 5.

45:15

And so, what I'm capable of doing is

45:16

getting it down about 40% about 60% of

45:19

the total work. Hopefully you guys see

45:20

when you have tasks that can actually be

45:22

broken down in this way, aka a task that

45:24

you can expand and run simultaneously

45:26

through some form of parallelization.

45:27

Just makes more sense to do all three of

45:29

these things simultaneously rather than

45:31

one parent agent being responsible for

45:32

everything like doing one and then doing

45:34

two, then doing three, then doing four.

45:36

What we can do is we can take two,

45:37

three, and four, stack them on top of

45:38

each other, add an additional step five

45:40

called a synthesizer, and then take the

45:42

results of these in and do it do it in

45:43

like a fraction.

45:45

Um another big reason is that agents are

45:47

what are called stochastic, aka they

45:48

don't always return the same answer. So,

45:50

if I ran, you know, Claude five times on

45:53

basically the exact same thing. Every

45:55

single time I have a slightly different

45:56

response, okay? Every time I have a

45:58

slightly different response. And just to

45:59

show you guys what I mean by that, I'm

46:00

going to open up um my Claude code over

46:02

here, and I'm actually going to open up

46:03

three different tabs. Let me just

46:05

visualize this. Stick this right in the

46:06

middle.

46:07

Okay, and then over here, let me just

46:09

make sure that all these are operating

46:10

the same. I'm going to say, "I'd like

46:12

you to determine five ways to improve

46:14

this code base."

46:16

I'm just going to paste this across all

46:17

three of these. I'll paste, and I'll

46:19

paste. Now, I'm just going to run all

46:21

three of them. And I just want you to

46:22

notice sort of what's going on here.

46:24

Obviously, the first thing it's going to

46:25

do is try reading the key files, but

46:26

check out the different solutions

46:29

basically that it's coming up with on

46:31

all three of these different runs. So,

46:33

in the first run, okay, uh breakage

46:36

broken image paths, missing title and

46:37

meta tags, nav links hidden inside

46:39

mobile with no replacement, project

46:41

cards aren't actually links, no keyboard

46:43

focus styles or script to content link.

46:45

The second was broken image paths,

46:47

missing meta tags, no mobile nav, but

46:49

now look, placeholder links everywhere,

46:51

typo in footer.

46:53

Okay, and you can see that, you know,

46:54

basically the more times we run these

46:57

you know, agents, and then the further

46:58

away they get from the beginning, the

47:00

more they tend to diverge. And there's a

47:01

physical reason for that, right? Like at

47:03

the very beginning, if this is sort of

47:04

like I don't know, the total answer. At

47:06

the very beginning,

47:08

you know, red is pretty similar to

47:11

black.

47:12

But eventually, it diverges a fair bit.

47:14

Green similar to red, and then it

47:16

diverges a fair bit. Blue similar to all

47:18

these, but it diverges a fair bit. And I

47:21

guess the point that I'm trying to make

47:21

is like, you know, over here, uh let's

47:24

pick another color so it's pretty

47:25

obvious that these are all a bit

47:27

different. We'll do purple. Over here,

47:29

these this is sort of like the zone of

47:30

similarity, right?

47:32

But then after you make it to a certain

47:33

point, because of the multiplicative

47:35

nature of how large language models work

47:36

under the hood, they're basically

47:37

multiplying the statistical

47:39

probabilities of like one token after

47:40

the other after the other after the

47:41

other, uh you have massive divergence in

47:45

the end result. And so, you know, this

47:47

might go ABC, this might go BCD, this

47:51

might go ABE, this might go ABZ, and

47:54

this might go ACQ, or something like

47:57

that.

47:58

What you can do is you can actually just

47:59

run five times. And now notice if I run

48:02

this once, I'd only get A, B, C. But

48:03

because I've run this another time, I

48:05

got all the way to D. I run this another

48:07

time, I got all the way to E. You know,

48:08

if you would just count up all of the

48:09

different unique answers here, I have A,

48:11

I have B, I have C, I have D, I also

48:13

have E, I even have Q, and then I have

48:15

Z.

48:16

So you can see here that like I'm

48:17

basically getting 2.5 times the total

48:19

number of possible answers by running

48:21

things multiple times and then just

48:22

averaging out and taking all the unique

48:24

outputs, right?

48:25

That's really the the principle of

48:27

stochasticity. Because they don't always

48:28

return the same answer, if you

48:29

parallelize your agents, you can

48:30

actually run multiple times with same or

48:33

similar queries.

48:34

Uh and then you can actually have

48:35

different answers given to you that just

48:37

sort of live outside of the distribution

48:39

uh or average run, which is pretty

48:41

amazing. So I'm going to show you guys

48:42

how that works specifically with um

48:44

debate and stochastic consensus models.

48:45

If anybody's seen my Agentic AI course

48:47

on that, you'll know more or less what I

48:49

mean by that. I'm also going to show you

48:50

some fan-in, fan-out um researching

48:52

flows as well as some some sequential

48:54

pipeline handoffs.

48:56

Uh but really the the fourth and final

48:57

reason is because model performance

48:58

degrades as context increases. So the

49:00

shorter and the cleaner your context

49:02

windows are, typically the better the

49:03

results are as well.

49:05

Uh what I mean by this is, you know,

49:06

because the parallelization aspect

49:08

typically involves sub-agents, which I'm

49:10

going to show you guys a little bit

49:11

about, you get to avoid the problem

49:12

where the increasing length, number of

49:14

tokens, um leads to poor performance.

49:16

And so you know, if like on average or

49:18

this is more or less the relationship

49:19

between the number of things in your

49:20

context window and then the performance

49:22

of the model, uh we are going to we're

49:24

going to end up just almost always

49:25

staying right around here, which is the

49:27

zone of good. By the way, I just made

49:30

that up. It's not actually called the

49:31

zone of good. Hopefully you guys

49:32

understand the distinctions there

49:33

though. Um when you parallelize and then

49:36

feed tiny chunks of a problem to

49:37

multiple agents, they can all be at the

49:39

zone of good. You don't actually have to

49:40

like go all the way down here. It's not

49:42

just one agent that's doing all the

49:43

work.

49:44

Okay, so so what are examples of how to

49:46

parallelize in the first place? Well,

49:47

there's like a built-in function called

49:49

agent teams now in Cloud Code, which

49:51

does a fair amount of this. So I'm going

49:52

to show you guys some ways to do that.

49:54

But, I just wanted to chat a little bit

49:55

more generally without even going into

49:57

agent teams first before I show you some

49:58

demos. Um of like different ways that I

50:00

personally approach problem-solving and

50:02

I've seen some of the best and the

50:03

brightest use CloudCode for this sort of

50:05

parallelization.

50:06

And I'm going to call them common team

50:08

patterns, okay? Um essentially, there's

50:11

three main things I want to cover. The

50:13

first is the ability to fan out and then

50:15

fan in. And so, that's where you

50:16

actually spawn a bunch of different

50:18

research sub agents and then you have a

50:20

synthesizer sub agent which takes all of

50:22

their outputs. And then, um based off of

50:24

the outputs of that synthesizer, you can

50:26

do either more fan out fan in flows or

50:28

you could do some form of final

50:30

synthesis step. Okay, so what I mean by

50:32

that is like let's say before you have a

50:34

query and it's, you know, I want to find

50:38

the best, okay, absolute best APIs

50:42

for my feature, whatever the feature is.

50:44

It's X feature, I don't know, it's like

50:46

some app that generates things,

50:48

whatever. So, I want to find the best

50:50

APIs out there for this feature that,

50:52

you know, allow me to very quickly and

50:53

easily do the things I want to do.

50:55

We can imagine like if you were to do

50:57

this um in the old-school linear path,

51:00

what would happen is CloudCode would

51:02

spin up, okay, in the same thread

51:04

research on site number one

51:07

and then go on site number two

51:09

and then go site number three

51:11

and then go on site number four, right?

51:13

And what would be occurring the entire

51:15

time that we're going through all these

51:16

different websites?

51:17

Well, the length of our total contacts

51:20

would increase, meaning our performance

51:22

on average would also decrease, okay? In

51:24

addition, it's taking time. So, it's 5

51:26

minutes here, it's 5 minutes there, it's

51:29

5 minutes there, it's 5 minutes there,

51:30

and so on and so forth. Then at the end,

51:32

what it would do is it'd have a final

51:33

synthesis step, which I'm just going to

51:34

call S, which would basically combine

51:36

one, two, three, and four together,

51:38

which could take a certain other amount

51:39

of time, maybe another 5 minutes, before

51:41

finally giving you your answer.

51:43

And so, the cost of the answer, okay, if

51:46

you think about it as like almost like a

51:47

line item.

51:48

The cost of the answer is, you know,

51:50

first of all, 25 minutes, which is

51:53

obviously non-preferable to instant, and

51:56

then, you know, a fair amount of tokens

51:59

on poorer

52:02

quality outputs.

52:04

You know, you're probably going to end

52:05

up spending a similar amount of tokens

52:06

regardless, but you're spending those

52:08

tokens on poor quality outputs because

52:10

you're kind of you're kind of down here

52:12

as opposed to up here, right? You're

52:13

you're here where you don't want to be.

52:15

Now, what fan out and fan in is is very

52:17

similar to what I showed you guys

52:18

earlier. You have a research query, and

52:20

that's, you know, find best APIs.

52:24

And so, what it does is Claude Code

52:27

basically goes in and then immediately

52:29

spawns,

52:31

okay? Let's just say

52:33

four research agents.

52:36

And so, now we have research agent one,

52:37

research agent two, research agent

52:39

three, and then we have research agent

52:41

four.

52:42

Okay, and so, what we're doing is here

52:43

is we're we're we're fanning out.

52:45

These all operate totally independently

52:48

accumulating their own context windows.

52:50

Because they're new agents, they're

52:52

almost always in the zone of good. Maybe

52:54

they'll push a little bit farther beyond

52:55

that, but they're still pretty good.

52:57

Once we're done with that, what we do is

52:58

we do the opposite, which is the fan in,

53:02

and we feed all of those into a final

53:04

synthesizer agent. That synthesizer

53:07

agent now has a different prompt. The

53:09

prompt is not, "Hey, go do this

53:10

research." The prompt is, "Hey, here's a

53:12

bunch of context from a bunch of other

53:13

models that have already done the

53:14

research." Meaning the prompt has to be

53:16

shorter. We then apply high-level

53:18

reasoning strategies and principles to

53:19

make that a synthesizer as smart as

53:21

possible, and say things like, "We want

53:23

you to integrate anything that overlaps,

53:26

as well as any outliers, and then score

53:28

them slightly differently." And so, you

53:30

know, rather than being all the way over

53:31

here with our big thing, you know,

53:32

probably we're somewhere over here in

53:33

the middle, which means the performance

53:34

is going to be a little bit better. And

53:36

then, obviously, the synthesis step can

53:37

occur in approximately the same amount

53:38

of time as the actual research, because

53:39

you you spawn almost an infinite number

53:41

of sub-agents to go do research for you.

53:43

And so really what happened now is you

53:44

have 5 minutes here, you have 5 minutes

53:46

here. You know, just add these up, it's

53:48

10 minutes. And so not only are we

53:50

significantly faster, we're also a lot

53:52

higher quality because now we have all

53:53

the the data and information laid out to

53:55

the synthesis agent. More importantly,

53:58

there are different models that are

53:59

better at different things. And so

54:00

within Claude, you have not only your,

54:03

you know, heavy lifter, which is usually

54:05

the Opus models, but you also have, you

54:06

know, your Sonnet models. And then,

54:08

although not a lot of other people use

54:09

them these days, you also have your

54:10

Haiku models. And so what you can do now

54:12

is for the research, which consumes a

54:14

massive number of tokens, but

54:15

realistically doesn't usually need like

54:17

a ton of reasoning for it, it's more of

54:18

like data extraction, you use something

54:20

cheap like Haiku and Sonnet. And then

54:22

for the synthesis, you use something

54:23

like Opus. Because you're applying

54:25

different models at different steps, not

54:28

only is it um going to going to occur

54:30

much faster cuz Sonnet works faster than

54:32

Opus, so maybe instead of 5 minutes

54:34

here, it's actually, I don't know, 3

54:35

minutes, but then the cost is going to

54:38

be a small proportion of the money that

54:40

you normally would have spent, just

54:41

because of the way that pricing on

54:42

Claude works, right? Pay attention here

54:44

to the fact that Claude Opus, you know,

54:45

in this case, 4.6 is 5 bucks, Sonnet 4.6

54:49

is 3. So we immediately save 60% right

54:51

there.

54:52

And that's just your base input tokens.

54:54

That's not taking into account um you

54:55

know, like the the the the massive

54:57

difference in also output token cost and

54:59

so on and so forth. And obviously things

55:01

get even better if you go down to Haiku

55:02

and and so on and so forth. And so you

55:04

can formalize this as a skill if you

55:05

would like, okay? I'm not going to. I'm

55:08

just going to feed it in a simple

55:09

prompt, but um this will illustrate what

55:11

I mean. Let's say I'm right over here in

55:13

my project, okay? Let me just delete

55:15

this um global cloud.md because we don't

55:17

need that anymore.

55:18

Then I'm going to essentially, let me

55:20

just go back here and then copy the

55:22

actual text.

55:24

As I use a fan out

55:26

fan in and researchers synthesizer

55:28

approach

55:30

to research the question

55:33

how best should I optimize this

55:35

codebase? Minimum five subagents, use

55:38

Sonnet to do the research and

55:40

contemplation.

55:43

Individual contemplation, Opus to

55:44

synthesize.

55:46

So now, what's going to occur is rather

55:48

than we us just waiting non-stop for all

55:51

of these, what this is going to do is

55:52

it'll fan out six Sonnet research

55:55

agents. Each are going to investigate a

55:56

slightly different optimization axis.

55:59

They're all going to focus on slightly

56:00

different things. Then they're going to

56:01

synthesize all of those results back

56:02

together with Opus. If I zoom out, you

56:05

can actually see all six of them running

56:06

simultaneously. D- despite the fact that

56:08

we're not using this agent team feature,

56:10

we're just using the sub agent feature

56:11

right now.

56:12

Uh you know, all of these things

56:13

basically immediately are generated.

56:15

Their contexts are quite short. So I

56:17

mean, in the grand scheme of things,

56:19

this is a much shorter context than we

56:20

would ultimately accumulate in our main

56:22

agent. All of them are focused on

56:23

slightly different things, which are

56:24

obviously autonomously managed by that

56:25

orchestrator. And then finally, these

56:27

six agents can finish in a linear amount

56:29

of time as opposed to, you know, like

56:30

multiple one.

56:32

So this just finished the architecture

56:33

research. It's going to wait for the

56:34

remaining five agents now. All right,

56:35

and it looks like it just finished all

56:37

six research runs. So now it's going to

56:40

synthesize all the findings with Opus.

56:42

It's then going to also be able to take

56:43

advantage of things like its planning

56:45

features and so on and so forth before

56:47

synthesizing.

56:48

And here it is, okay? High impact easy

56:50

fixes gives us a big list. It's also

56:52

writing the high to medium impact easy

56:54

to medium effort. And so, I mean, you

56:57

know, obviously I'm not just pulling

56:58

this out of my my ass here. Anthropic

57:00

has done a lot of research on the best

57:02

way to solve problems. And you know,

57:04

Opus with a bunch of Sonnet sub agents

57:06

massively outperforms Opus both on time,

57:09

but then also quality. Specifically

57:10

because of, you know, Sonnet's longer

57:12

context window as well as just like

57:14

general usability.

57:15

That's what I care about. I just care

57:16

about my own usability here. I could

57:18

spend as much money as I want on these

57:19

things at this point. What I care about

57:20

is like, how can I extract the maximum

57:22

quality with the minimum amount of time?

57:24

And that's the design pattern that you

57:25

want to use. So I mean, like use this

57:27

anytime you're contemplating problems.

57:28

And you don't just have to contemplate

57:29

like specific API problems or

57:31

development problems as well. Like I use

57:33

stuff like this anytime I'm designing

57:35

business systems, anytime I'm designing

57:37

process optimizations. I mean, I did

57:39

this the other day when I was doing

57:40

product differentiation, basically

57:41

coming up with different ways to price

57:43

and package products for a company that

57:45

I now own that does this sort of thing.

57:47

The opportunities here are basically

57:49

limitless. You do this for competitor

57:51

research. You can do this for whatever

57:52

the heck you want.

57:53

And I I commonly apply it. Okay? So,

57:55

that's fan out and fan in where you

57:57

basically spawn and researchers, usually

57:59

using a cheaper, dumber model like

58:01

Sonnet. And then you have a a larger

58:03

synthesizer model that actually combines

58:05

the results. And that's how you get, you

58:07

know, some of the best quality and then

58:09

also the best quantity. Next, I want to

58:10

chat debate and stochastic consensus

58:12

because it's kind of simpler similar,

58:14

but you know

58:17

it's [clears throat] also a little bit

58:17

different. I use debate and stochastic

58:19

consensus to basically like hammer out

58:21

nuanced arguments and nuanced quality

58:24

discussions. You know earlier how I said

58:26

we had one agent come up with ABC,

58:28

another one come up with CDE, another

58:30

one come up with like ABQ, and so on and

58:32

so forth. Well, basically with

58:34

stochastic consensus and then later

58:36

debate, what we're doing is we're having

58:38

different sub agents come up with

58:40

different lists of solutions.

58:43

And then we have something else go

58:44

through, identify all of the mode

58:48

identify the mode, which is the

58:51

frequency of, you know, the the number

58:53

of times that a solution pops up. So,

58:55

let's say solution A pops up twice.

58:58

Okay, this synthesizer agent would say

58:59

okay, there's two A's.

59:01

B pops up twice, so we go two B.

59:05

C pops up twice, so we go two C.

59:08

D pops up how many times?

59:11

One, so we go D.

59:13

Then counts E, then also counts Q. And

59:16

so in this way you could see

59:17

statistically speaking, you know, a lot

59:19

of agents think these three are great

59:20

solutions. One agent thought this was a

59:22

good solution. Another agent thought

59:24

this was a good solution. And finally,

59:25

another agent thought this was a good

59:26

solution. Basically, the votes of

59:28

confidence here are fewer. And then what

59:30

you can do is you could use this is

59:31

almost like

59:32

like a weighted average to tell you what

59:35

approach to take. You know, if it's like

59:37

an equation where like my final I don't

59:39

know decision which will just say

59:41

decision is kind of like this it would

59:43

equal 2A + 2B + 2C + D + E + Q. I know

59:49

this is math but don't get scared here.

59:51

The point is not to actually calculate

59:52

the final solution. The reality that I'm

59:55

attempting to convey to you is that

59:57

because so many models came up with A,

60:00

so many other models came up with E and

60:02

B and Q and so on and so forth.

60:04

You can quickly determine consensus

60:07

between a number of agents that come up

60:09

with ideas. And then you can also

60:11

determine which ideas are genuine

60:13

outliers in so far that you know only

60:15

one out of three models actually came up

60:17

with this thing. One out of 24 models

60:19

suggested you should do X Y and Z thing.

60:22

And so you get to farm both like the

60:23

statistically most likely answers to

60:26

solutions but also like the massive

60:27

outliers which can make you quite

60:30

I want to say competent at solving

60:32

problems in a very short period of time.

60:34

And this works in a really similar way

60:35

to what I talked about earlier with like

60:37

the total solution space, right? You

60:39

know, if there are really a fixed number

60:41

of ways to solve something and the

60:43

reality is there are a fixed number of

60:44

ways to solve something and there also a

60:46

certain number of ways not to solve

60:47

something. Well, what you want to do is

60:49

you just want to like cover that ground

60:50

as quickly as possible. And in reality

60:52

what you could do is you could quickly

60:53

spin up an agent to do all of the figure

60:55

out all the ways not to do something,

60:57

okay? And then you could have you know

60:59

one sub agent slowly figuring out no

61:01

this doesn't work. No this doesn't work.

61:03

No this doesn't work all simultaneously.

61:05

And then what you're end what you end up

61:07

with is you just end up with like this

61:08

beautiful field of like highly

61:09

differentiated green which tells you

61:11

what you can actually do.

61:13

And I understand this is more conceptual

61:14

but just bear with me here. I'll show

61:15

you guys an actual example in a moment.

61:17

Now stochastic consensus is cool it's

61:19

sort of like a first go but debate is

61:21

even cooler because now what you do is

61:23

you basically take all of these points,

61:25

okay, and then you feed them into an

61:27

open like conversation or chat room

61:30

where all other models can weigh in on

61:33

solutions that might not actually be

61:34

very obvious. So now, okay, if I just

61:37

recreate the solution, we have agent one

61:39

come up with ABC, agent two come up with

61:42

BC, I don't know, let's just say E,

61:44

agent three come up with ABQ.

61:46

Okay? What we do is we divide this into

61:49

time steps. And so this is time one,

61:51

this is time two, this is time three,

61:54

and this is time four.

61:55

What we do at every time step is we

61:57

allow all other agents to look at all of

62:00

the uh conversations and and all the

62:02

thoughts that all the other agents have

62:04

had, okay? And what occurs as we move

62:06

through is agent one gets to see agent

62:09

two and agent three's responses. And so

62:10

it gets to differentiate. Maybe now it

62:12

goes ABC E Z, because it can comes up

62:16

with some additional solution by

62:17

comparing its to, you know, two and

62:19

three. Maybe this one comes up with with

62:21

BC, but then it eliminates E because it

62:23

just doesn't think that made too much

62:24

sense, and then it comes up with an F.

62:25

You know, this one comes up with with uh

62:27

I don't know, two different letters, and

62:29

then ends up uh you know, also

62:30

identifying some of the previous

62:32

solutions, but then combining them in

62:33

new ways and stuff like that to come up

62:34

with better ones.

62:36

And so what we do with the debate is

62:38

it's not really a debate in the

62:39

practical sense. It's not like, "Hey,

62:40

your job is to try and convince other

62:42

people why A, B, and C are the best

62:43

solutions."

62:44

What it is is every model has access to

62:46

all of the other models. And so because

62:48

they have access to all of the other

62:49

models, and they don't have to spend all

62:50

the time reasoning, they can just see

62:51

the results, they can then incorporate

62:53

those and come up with increasingly

62:55

nuanced uh solutions and you know,

62:57

ultimately span a large search space in

63:00

a very short period of time. And so we

63:02

can just proceed with this all the way

63:03

down. You can run as many of these like

63:05

steps as you as you want until

63:06

ultimately you have like a list of

63:08

solutions provided by a bunch of

63:10

different models that are just way more

63:11

complex, way more nuanced, and also just

63:13

like way more interesting than the

63:15

initial ones that, you know, one agent

63:17

might have come up with. All right, so

63:18

I'm back in my business workspace here

63:20

and uh we're still doing research on

63:22

tomatillos, but I thought this is

63:23

actually a pretty good example. Why

63:24

don't we use stochastic multi-agent

63:26

consensus to come up with all of the

63:27

different ways you can make a sauce

63:28

using a tomatillo? Use stochastic

63:30

multi-agent consensus to determine all

63:32

of the different ways that you could

63:33

make a nice-tasting sauce using

63:36

tomatillos. I want every agent to come

63:38

up with at least 10 independent

63:40

responses, then have them synthesized

63:42

and turned into just a giant list of all

63:44

of the possible things you could do.

63:47

So, what this skill, stochastic

63:48

multi-agent consensus, does, if I open

63:50

it up, is basically it breaks down a

63:52

query into n other queries, that's where

63:56

it says "spawn n agents", with the same

63:59

or a slightly different prompt to

64:01

independently analyze a problem, then

64:03

aggregate results by consensus, which

64:05

you use for decision-making, ranking

64:07

options, strategic analysis, or any

64:09

problem where you want to filter

64:10

hallucinations, and then surface what

64:12

are called high-variance ideas.

64:14

So, anytime I use the word consensus,

64:16

poll agents, stochastic consensus, spawn

64:18

n agents, so on and so on and so forth,

64:20

it'll go and it'll it'll do the thing.

64:22

So, just scrolling down here, you can

64:23

see that it read through the skill and

64:24

it spawned 10 agents all looking at

64:27

slightly different angles here.

64:29

And, you know, these are very similar

64:30

prompts.

64:31

Brainstorm all the different ways you

64:32

can make a nice-tasting sauce using

64:34

tomatillos. This one's here, brainstorm

64:36

all the different ways you can make a

64:37

nice-tasting sauce using tomatillos.

64:39

This one here, brainstorm all the

64:40

different ways you can make a

64:41

nice-tasting sauce using tomatillos.

64:43

But, the idea is, you know, one is a

64:44

conservative tradition-minded chef, the

64:46

other is an adventurous boundary-pushing

64:48

chef, the other challenges conventional

64:50

wisdom, the other reasons from first

64:51

principles, and so on and so on and so

64:52

forth.

64:54

Now, because, you know, this is a pretty

64:55

simple and not very intellectually

64:56

difficult exercise, all 10 agents have

64:57

actually already already finished. And,

64:59

you can see that I was able to scan a

65:01

massive search space in a very short

65:02

period of time, despite the fact that

65:04

this problem was pretty simple. So, what

65:06

it's doing is similar to what I showed

65:07

you earlier with those n researchers and

65:09

then having some sort of synthesizer

65:11

model, what this is now going to do is

65:13

indeed duplicate the outputs and then

65:15

give me a list of pretty nuanced answers

65:17

that realistically scanned most of the

65:19

search space in a very short period of

65:21

time. I'm sure you can imagine you could

65:22

scale this up if you had like some sort

65:24

of dedicated infrastructure, whether

65:25

it's a local model or something like

65:26

that. You could theoretically have stuff

65:27

like this running all the time just

65:29

ideating and coming up with new

65:30

approaches to solve on long-standing

65:32

problems. And this is actually the exact

65:33

way that I don't know if you guys have

65:35

seen,

65:36

uh you know, they're throwing Opus now

65:37

or GPT 4. or GPT or other models at like

65:42

these big math questions and asking them

65:43

to solve them. This is exactly how

65:45

they're doing them all under the hood.

65:46

So, as you guys could see, we've pulled

65:47

10 agents. There are 119 raw ideas.

65:51

Counting for duplication, there 52 in

65:53

total that are new. So, what we're going

65:55

to do is we're actually going to look at

65:56

this consensus report and then

65:57

ultimately its answers. All right, we

65:59

have the consensus report. Opening it up

66:01

here, you can see there are 52 total.

66:04

The first is salsa verde, cruda. The

66:06

next is tomatillo avocado crema. The

66:08

third is agua chile verde and so on and

66:10

so on and so forth. I could work my way

66:11

all the way down here. A bunch of

66:13

different types. You know, I could have

66:14

had one agent come up with all of these.

66:15

I could, okay? But uh the probability

66:18

that it would have been able to one come

66:19

up with like a highly differentiated

66:21

list like this and two scan as much of

66:23

that search space in the same amount of

66:24

time is very low. And so, I'm sure you

66:25

can imagine you can apply this to any

66:27

business problem that you guys are

66:28

currently having to just come up with a

66:29

bunch of low-hanging fruit solutions as

66:31

well as like unique and and and outlier

66:33

solutions as well. We even have like

66:35

Indian-influenced sauces,

66:37

Persian-influenced sauces,

66:38

Caribbean-Latin fusion sauces and so on

66:40

and so forth. An outlier that I'm

66:42

definitely not trying anytime soon is

66:44

tomatillo beurre blanc, which is French

66:46

butter sauce using tomatillo's pectin as

66:48

a natural emulsifier. No, thank you. So,

66:50

what would debate look like? Debate is

66:51

more or less the exact same idea. Um in

66:53

my case, I just turned this into a

66:55

skill. It's called model-chat.

66:57

Basically, what occurs is we spawn five

66:59

cloud instances in a shared conversation

67:01

room where they debate, disagree, and

67:03

converge on solutions. We use

67:05

round-robin turns with parallel

67:06

execution within each round that

67:08

triggers on terms like chat and so on

67:10

and so forth. So, what I'm going to do

67:11

here is I'll say, "Great, this looks

67:12

awesome. I'd like you to rerun this, but

67:14

with model-chat. Make sure at least 10

67:16

agents are having conversations about

67:17

this, and then uh you know, if any of

67:19

the sauces just sound insane or terrible

67:22

or crazy, then obviously have them

67:23

discuss that as well." Just like our

67:25

stochastic multi-agent consensus took

67:27

advantage of like time, basically, and

67:29

traded it off against total tokens,

67:31

we're doing the same thing. So, what

67:32

we're going to do is we're going to

67:33

start by extracting from the user's

67:35

message from the user's message the

67:36

topic or problem, the mode, the number

67:39

of agents, and the number of rounds.

67:40

It's then going to run an actual script

67:42

that I've set up here that's automates

67:43

the process of like having each of the

67:45

agents look at each of the other agents'

67:47

responses before finally doing a

67:48

synthesis. Speaking of which, I just

67:50

read through a couple of those, and I'm

67:51

actually just going to make some

67:52

tomatillo sauce right now, so I'll be

67:53

right back. Okay, so me looking at the

67:56

conversation over here, just asking it

67:58

to like give it to me.

68:00

You actually see that all the agents are

68:02

doing some thinking, and the contrarian

68:04

is starting with 15 ideas. It'll

68:07

immediately challenge the ideas that

68:08

deserve it. They're now listing their

68:10

disagreements. So, does this actually

68:12

work? Is it a structurally sound

68:13

technique or a restaurant stunt with an

68:15

unacceptable failure rate? Is tamarind

68:17

redundant or complementary? You know,

68:19

does tomatillo chocolate belong on the

68:21

list? If so, where? Should mole verde be

68:24

in tier one or tier two? So, they're

68:26

having discussions on an ongoing basis,

68:28

which is always really fun to watch that

68:30

we can monitor and then obviously

68:32

synthesize into an answer. Okay, and

68:33

then finally we have the tomatillo

68:35

synthesis over here. Um tomatillo's

68:38

pectin content is under appreciated.

68:41

Tomatillo husk tea, unfortunately, is

68:43

not cool. The foundational tier is

68:45

settled and non-negotiable. And then I

68:46

actually look at the foundational tier,

68:48

you can see we actually have a bunch of

68:49

different highly recommended sauces.

68:52

Again, some of these are very like

68:54

nuanced. Lacto-fermented tomatillo hot

68:55

sauce, Takis squeeze bottle drizzle,

68:58

enchilada sauce, tomatillo aguachile,

69:00

and so on and so on and so forth.

69:02

And you know, I this is just a really

69:03

shitty example, but hopefully you guys

69:05

understand that you can take this to

69:06

more or less anything that you want.

69:08

Whether it's, you know, designing a new

69:09

computer programming approach to a

69:11

particular problem, whether it's

69:12

choosing the right framework to approach

69:14

a or tackle a task with or something

69:17

else. Okay, so I just did all of the

69:18

previous example using a pretty

69:20

straightforward, you know, like dietary

69:23

or chef sort of example. But now I want

69:25

to use this on an actual app and really

69:27

just have all of these different models

69:29

discussing things and doing so in a very

69:31

short period of time. What I have here

69:33

is I have like an algorithmic art

69:35

example. And this is actually something

69:36

that Claude developed. It's part of

69:37

their algorithmic art base skill, which

69:39

I think is actually like applied or

69:41

supplied, I should say, in

69:44

the Anthropic Anthropic skill directory.

69:47

You can adjust some things like the the

69:48

stroke weight and like the damping and

69:50

so on and so forth and actually have it

69:52

like come out with very unique designs.

69:54

You can then just like save the image

69:56

and then boom, now you have like a cool

69:57

like wallpaper or something like that.

69:59

It's kind of neat. But I want to I want

70:00

to improve this as much as humanly

70:01

possible. And the reason I'm doing it

70:03

like this is because I also want to show

70:04

you guys how to apply the same

70:05

approaches that I just showed you to

70:06

agent teams instead, which are obviously

70:09

a much more streamlined version of doing

70:11

the exact same things that I've done so

70:12

far. It's just streamlined in the sense

70:14

that, you know, it is built out of the

70:16

box to do everything, but it does so at

70:18

the cost of some tokens. So I'm just

70:20

going to go back over here and then I'm

70:22

just going to look at

70:22

synaptic-drift.html

70:24

within art. I just need to make sure to,

70:25

you know, remember what folder that's

70:27

in. Then I'm just going to open up

70:28

another Claude instance. Now, a lot of

70:30

the advanced stuff, as we know, is

70:31

actually only available in the terminal

70:33

and I think agent teams are a lot better

70:34

managed in the terminal. So I'm just

70:35

going to open up the terminal.

70:37

I'm going to full screen it here as

70:39

well. Let me delete that and then go

70:40

full screen.

70:41

And you know, I could do it in here. I

70:42

could also do it in like ghost TTY,

70:45

which is probably my favorite like

70:46

terminal to use within Claude, but for

70:49

now, you know, I I have my agent teams

70:51

idea. So I'm I'm basically now going to

70:53

say, "Hey, I'd like you to optimize

70:56

synaptic-drift.html

70:58

and turn it into a full-fledged

71:00

application. However, rather than just

71:02

do this all naively yourself, I want you

71:04

to take advantage of stochastic

71:05

multi-agent consensus. I want you to

71:07

take that skill and then apply it using

71:09

the agent teams feature.

71:11

You'll orchestrate a team of agents that

71:13

do all of this stuff. Don't just use

71:15

what's in the skill itself because

71:16

that'd be running it a little too

71:17

simply. I actually want you to to read

71:19

through the whole skill and then use

71:20

that to spawn agent teams. Okay, so it's

71:22

going to start by reading the skill def

71:23

and then the HTML file itself, which

71:25

it's found in art. It's then going to go

71:27

and read through the agent teams tooling

71:29

and everything that it needs in order to

71:30

basically spin this up easy.

71:33

So, it'll start by creating a team for

71:34

the consensus workflow.

71:36

Spawning 10 analyst agents with

71:38

different framings, then finally

71:40

aggregating their recommendations and

71:41

implementing the winning features.

71:43

So, the very first thing it's going to

71:45

do is spawn the analyst agents. And you

71:46

can see now the UX has changed a little

71:48

bit. See down at the bottom where I have

71:49

these different analysts that are

71:50

running? So, if I go shift down, I can

71:53

actually see all of their different

71:55

stochastic multi-agent um kind of

71:57

consensus threads.

71:58

So, now they're all spawning and running

72:00

in parallel, which is pretty neat. At

72:01

any point in time, I could press enter

72:03

to view sort of the conversations and

72:05

and what they're doing. And I should say

72:06

I I should note that um

72:08

you know, the stochastic multi-agent

72:09

consensus applied to agent teams is

72:11

basically just the debate built in

72:12

because the agents actually can

72:13

communicate. Um the team lead can also

72:16

orchestrate that communication, too. So,

72:18

you know, it's not actually um really

72:19

independent, which is neat.

72:21

Uh you could spawn all of these in like

72:23

different windows if you want to. You

72:24

can also just con- continuously hold

72:26

shift and then go up and down to select.

72:28

What I'm doing is I'm just reading

72:29

through a bunch of different threads and

72:30

conversations. And it's clear they all

72:32

start by just reading through

72:33

synaptic-drift.html.

72:35

Um finally, uh you know, this is now

72:38

returning a bunch of

72:39

agent conclusions back. And more

72:41

importantly, it's also coming up with

72:42

consensus, which is nice. All right,

72:44

what it's going to do is just take all

72:45

these now and close them down while also

72:48

um looking at the consensus, the bugs,

72:51

the divergence, and then ultimately

72:53

outliers.

72:54

So, the consensus recommendation of our

72:56

next feature is high-res exports, a

72:58

preset system, URL state and shareable

73:00

links. The bugs are the race condition

73:03

and regenerate, download saves

73:04

mid-render, PG height not checked.

73:07

Uh divergence is one or sorry, six out

73:09

of 10 agents suggest debounce

73:10

regeneration versus a live preview.

73:13

Then the outliers have also come in,

73:15

mobile responsive layout, live animation

73:17

mode, seed history, web worker offload,

73:19

mouse attractor, repeller, and kill

73:21

sidebar overlay. So, this is all really

73:23

cool. You can see now it's coming up.

73:24

It's actually just deleting my uh old

73:26

tomatillo stuff. I guess we happen to be

73:28

using the same file or something.

73:29

Instead, it's coming up with this giant

73:31

list of different conditions and

73:32

features that it can build. Okay, now

73:34

it's actually shutting down all the

73:35

agents, implementing it. Um just because

73:37

I want this to do so faster, I'll say

73:39

use agent teams to do the

73:41

implementation. And you can see it's

73:42

actually gone through here and then um

73:44

added all of what we needed in order to

73:46

implement the tool the features that the

73:48

model suggested. In addition, it's also

73:50

spawning review agents to see if we can

73:52

improve the quality of the generated

73:54

code, spot problems, and stuff like

73:55

that.

73:56

So, if I go shift down, I can see all

73:58

those. So, we have now reviewer bugs,

73:59

reviewer features. Let's just see what

74:01

reviewer bug says. Okay, it's now

74:02

sending the review to the team lead, so

74:04

it's communicating that back, taking a

74:06

look at what the reviewer is saying. Now

74:07

that it's opening it up, you can see we

74:09

now have a ton more features. We have

74:11

different presets, so ocean drift, ember

74:13

storm, ink wash, neon plasma, neural

74:16

fire.

74:17

With the ability to modify colors, we

74:19

have 1x, 2x, and then 4x downloads,

74:22

which I don't think you guys could see

74:24

because my face is in the way. But if

74:25

you just um

74:26

look down over here, you see there's

74:28

significantly more functionality. Um we

74:30

can download a PNG at 4x as well.

74:33

We have simple like space bars to reload

74:35

and change things. We could change the

74:37

the speed and so on and so forth. Um

74:39

ultimately, this is just a better app,

74:40

right? And so, we did this by basically

74:42

just exchanging a couple of my dollars

74:44

and tokens

74:45

for, you know, a bunch of different

74:46

agents all coming up with their own

74:48

ideas and then ultimately executing on

74:50

them. Hopefully, you guys could see you

74:51

can apply the same approach to more or

74:52

less anything. There are obviously

74:54

optimal token tradeoffs, but um when you

74:56

spawn the sub agents that are a little

74:57

bit less capable, um like Sonnet versus

74:59

Opus, typically that math works out and

75:01

you end up being able to do just as much

75:03

if not more in a shorter amount of time

75:05

for less money. All right, and then

75:06

finally, pipeline, which is sequential

75:08

handoff between specialists. I mean, I

75:09

just showed you guys a little bit of

75:11

that earlier with um agent teams sort of

75:13

spawning review bugs and stuff like

75:14

that. But basically, that's more or less

75:16

it. You have task A done by some agent,

75:19

which is specialized for task A. You

75:21

then pass that off to agent B, which is

75:23

specialized for task B, and then

75:25

ultimately, agent C, which is

75:26

specialized for for task C. And so, I

75:29

mean, like you could just have A do all

75:31

three of these things. The issue with

75:33

having A do all three of these things

75:34

though is one, um if you guys remember

75:36

earlier, good lord, this is getting a

75:38

little messy. You know, we're no longer

75:39

in the zone of good because odds are it

75:41

has like tons of context from literally

75:43

everything that it's done before. So,

75:45

you know, like it would have started off

75:46

over here, and that would have been

75:47

okay, but now it's over here, and then

75:48

now it's over here.

75:50

And then two, like you you

75:52

sometimes

75:54

fast and and and good development is

75:56

often at odds with like really in-depth

75:58

testing, let's say. And so, if you think

76:00

about it conceptually, like a a

76:01

developer agent will have different

76:03

incentives than like a testing agent.

76:05

The developer agent will be incentivized

76:07

to like build things that works really

76:09

quickly using, you know, whatever is

76:10

available to it. Whereas the testing

76:12

agent will be incentivized to try and

76:15

like spot all of the issues. And so,

76:17

like building things new is sort of at

76:19

odds with like repairing the old things.

76:21

And in that way, if you try and have one

76:23

agent do everything, the probability

76:25

that it will be able to do it as well as

76:27

possible versus if you just spun up

76:28

specialized agents that were like highly

76:30

tuned for that thing,

76:31

assuming their intelligences are all

76:33

held equal here. I'm talking about like

76:34

nonstop Opus calls, not Opus Sonnet, and

76:36

so on and so forth.

76:38

As is is definitely different. So, my

76:40

recommendation would be, you know, like

76:41

what I would do is I'd have like a dev

76:43

agent for A like I just did. Then I'd

76:45

have some form of like bug fix for B.

76:48

Then I'd have some sort of like test,

76:50

maybe bug and Q&A. And I'm not going to

76:52

redo that example because one, I want to

76:53

be respectful of your time, but two, I

76:55

just showed you that exactly with the

76:56

agent teams example.

76:58

Um I guess the meta example here is you

77:00

combine all three of these and then um

77:02

just have all of them interacting

77:04

constantly for best results. Like you

77:06

have, you know, debate and stochastic

77:07

consensus to come up with like the best

77:09

ways to you know, improve on a product.

77:11

Then maybe you do some fan out fan in

77:13

and researchers to go look at like

77:15

different APIs and different design

77:17

patterns that you could use to fulfill

77:18

that before finally handing that off to

77:20

some sort of like bug reviewer, QA, or

77:22

tester. Uh but hopefully it's clear that

77:24

yeah, all of these things do not exist

77:25

in isolation. Uh they all exist

77:26

together.

77:28

Next, let's talk context management,

77:31

which put really simply is just all of

77:34

the files and folders and organizational

77:37

methods that you put into a workspace to

77:39

allow Claude code to effectively manage

77:42

whatever work you have.

77:43

Now, I'm seeing a lot of people try and

77:45

delegate work right now, sort of like

77:48

human companies do with CEOs, you know,

77:52

CTOs, CMOs,

77:55

uh Claude coder agents, and software

77:57

engineers, and stuff like that.

77:59

And I think initially when I looked at

78:01

this, this one's called paper clip

78:02

specifically, it's got a pretty

78:04

interesting repo that you can check out

78:05

right over here. It's all about running

78:06

your whole business with our agent team.

78:09

I think initially it's really easy to

78:10

look at these and be like, "Hey, this is

78:11

stupid." You know, I mean, that's that's

78:13

what I did. I uh made a couple of videos

78:15

and I talked uh ad nauseam with a couple

78:16

of my friends and I was like, "This is

78:18

dumb. Why would we try and fit agents,

78:19

which think very differently than human

78:21

beings, into the exact same

78:22

organizational hierarchies we've been

78:23

using for the last 150 years?" Just

78:25

doesn't make sense. Human brains are

78:26

different than agent brains. The latter

78:28

is obviously a lot more spiky and good

78:29

at certain things while sucking at

78:30

others.

78:31

But anyway, so as as quick as I was to

78:33

initially dismiss this idea, what What

78:35

come to realize is that subagents, as

78:37

these org charts

78:39

and skill.md files, which as we know are

78:42

self-contained SOPs that exist within a

78:45

file, capitalize as skill.md.

78:48

These are actually just two flavors of

78:50

the exact same thing.

78:52

What they are is they're just different

78:54

ways of organizing your markdown files.

78:56

And so, just like in my case, we ran a

78:59

model-chat skill earlier for me to show

79:01

you guys how, you know, models debated

79:02

and stuff like that, okay? We had a

79:04

skill.md within it that stored a bunch

79:06

of information that was like

79:07

hyper-specific to that skill. We had

79:09

model-chat.py,

79:11

which was a tool that the skill could

79:13

use.

79:14

So too are subagents organized in

79:15

basically the same way. I guess what I'm

79:17

trying to say is like, okay, if we take

79:19

subagents on the left-hand side, what

79:21

was one of the main reasons why we like

79:22

using subagents, okay? It's because it's

79:25

a clear or fresh

79:28

context window, right?

79:30

All right, awesome. So, that's one.

79:32

How about the fact that it's

79:34

specialized?

79:35

Awesome, that's another.

79:37

How about the fact that the subagent is

79:39

probably more reliable at subagent

79:41

specific tasks? Right, that's another

79:42

one. And then how about the fact that

79:43

it's written in, you know, markdown

79:45

format

79:46

with tool use?

79:48

Well, fantastic, that's another one. If

79:50

we look at like how that equates to

79:51

skills, honestly, the only thing that's

79:55

missing is the fact that the context

79:57

window is not entirely clear or fresh.

80:00

But, you know, what you can do with

80:01

these

80:03

is because skills are so efficiently

80:04

written, um they're basically a form of

80:06

compression that pushes you towards a

80:09

shorter context window anyway.

80:11

So, basically, the only real difference,

80:13

if I'm honest, and keep in mind like

80:14

when you instantiate a subagent, you're

80:15

giving it, you know, a a little prompt,

80:17

right? Kind of similar to the way a

80:18

skill works. The only real difference

80:20

between the two is just the amount of

80:21

context in the subagent versus the

80:22

skill. But, I want you guys to know that

80:24

subagents are honestly basically skills,

80:27

and skills are basically subagents.

80:29

They're just slightly different ways of

80:30

storing information.

80:31

So, why am I bringing this up? Um just

80:32

because I'm coming to realize that the

80:34

two are the two are very similar and

80:35

they're soon I'm sure in the future

80:36

going to be like merged even more so

80:38

into a similar concept. Um all these two

80:41

point at are just different ways of

80:42

organizing your contacts and basically

80:44

organizing the way that you you get

80:46

tasks done. One delegates via CEO to

80:49

CTO, CMO, CTO, all all this stuff,

80:51

right? I don't know why there's two CTOs

80:53

now that I'm looking at. That's kind of

80:54

weird. Whereas the other one stores

80:56

things in a skill done in me. Like just

80:57

going back to anti-gravity right over

80:59

here, right? Like I can go to the skills

81:01

folder and then I could go and find that

81:03

model-chat. And I mean like the way that

81:04

this is written is basically the exact

81:06

same um you know, schema basically that

81:09

a subagent is written in. You know, if I

81:10

go over here to Claude Code's actual

81:12

documentation page on subagents, I mean

81:15

you actually have basically the exact

81:17

same structure. See how here it says the

81:19

title code reviewer, description,

81:21

prompt, tools, model, okay? You see over

81:23

here, what do we have? We have the name,

81:25

okay? We have a description and then we

81:26

also have the tools. I mean like the

81:28

model is sort of baked in here because

81:29

it's in our main thread. It's going to

81:30

be Opus 4.6. But you guys are seeing

81:32

like skills and subagents are actually

81:33

really similar. They're just slightly

81:34

different ways of organizing

81:35

information.

81:36

So, I'm making this big point because I

81:38

think that's important to realize as we

81:39

continue moving forward with Claude Code

81:41

and other tools and we get more and more

81:43

advanced with them. Um the shapes of how

81:45

we're transmitting information to our

81:46

models will likely end up being quite

81:48

quite similar. Whether one person

81:50

decides to use a paperclip style big

81:52

fleet of agents that does XYZ, which

81:54

maybe you know, just a couple of months

81:55

ago I might have looked at, scoffed and

81:57

said like, "Well, that doesn't do

81:58

anything." Um you know, so too are

82:00

skills basically the the same thing. So,

82:02

the model intelligence is growing more

82:04

and more and more capable within the

82:05

harness, which is what allows the the

82:07

development of these really interesting

82:08

organizational hierarchies. So, what are

82:09

some of these organizational

82:10

hierarchies? Well, I've already shown

82:11

you paperclip here. And the way that

82:13

paperclip works or it's rather it's

82:14

supposed to work is this is like a

82:16

dashboard which um somebody develops

82:18

that you know, I think just preys off of

82:20

maybe prey isn't the right word, but it

82:21

uses people's misunderstandings of how

82:23

agents work um and equates them and

82:25

anthropomorphizes them and makes them

82:26

seem really similar to humans and then

82:28

it puts us in front of you so that you

82:29

feel like you're running a whole team.

82:31

And so in this way clearly it's broken

82:32

down by role, right? Whereas the average

82:34

skill is not broken down by role, the

82:36

average skill is broken down by

82:37

function.

82:38

Um also skills typically don't delegate

82:40

to other skills. That's really the main

82:42

difference. But paperclip isn't the only

82:43

one that's like this. Here's another

82:45

good example, company helm. This one

82:47

over here is a very same similar sort of

82:49

idea where you basically have an AI

82:51

studio. Within the AI studio you define

82:53

a bunch of different roles for your

82:55

agents and so on and so forth and then

82:56

that's ultimately what allows you to

82:58

manage your projects. This instead of

82:59

being left to right is obviously, you

83:01

know, organized a little bit differently

83:02

with a front-end builder, a QA runner

83:04

and and so on and so forth. How about

83:05

open goat, which is the AI autonomous

83:08

organization of open claw agents? Again,

83:10

you know, it's doing this with like CEO,

83:12

head of sales, customer support based

83:14

organization, which I don't really

83:15

believe is ideal. I don't really think

83:16

you should have this level of direct

83:18

reports. I mean like think about a why.

83:20

All of these could just be Opus 4.6.

83:21

They could be way smarter. They could

83:23

pull from some sort of shared context

83:25

pool and I think you really wouldn't

83:26

leave that much out. Uh but it is an

83:27

interesting approach.

83:29

This one over here is called the system,

83:30

which is obviously using some sort of AI

83:33

generated diagram here. But it's 26

83:35

specialized AI agents which we've

83:36

thought about that do architecture,

83:38

design, product development, release,

83:41

operations, and so on and so forth. This

83:43

one over here I think is called Gas

83:45

Town, which is basically where you have

83:47

a mayor, which is your AI coordinator, a

83:49

bunch of different crew members, and

83:50

then also polecats or worker agents. You

83:52

guys may have heard of Crew AI. It's the

83:55

same sort of idea. It's a fast and

83:56

flexible multi-agent framework which

83:58

supposedly delegates things. Okay, where

84:01

you have crews that have different

84:03

agents within them, each with their own

84:05

segregated tool calling and stuff like

84:07

that. You know, it's another way of

84:08

organizing information. This one over

84:09

here, Swarm Claw, is CEO based,

84:12

developer, researcher, and again you

84:14

have delegation. So all these are

84:15

different attempts by different groups

84:17

of people to try and determine like the

84:19

best organizational hierarchy of agents.

84:22

And I think pretty much all of them suck

84:23

right now, to be clear.

84:25

Um but I I just want you guys to know

84:27

and level with me that these are just

84:29

different ways of organizing

84:29

information. Just like you have skills,

84:32

and skills are highly, you know,

84:33

specific to you, it's just a collection

84:35

of markdown files with names,

84:36

descriptions, allowed tools, and then

84:38

like SOPs. Sub-agents are basically the

84:40

exact same thing. So, as the field

84:42

continues to mature and there are better

84:44

and more novel context management

84:46

strategies out there, um multi-agent

84:48

orchestrators, essentially, uh you know,

84:50

these things will grow more and more

84:51

differentiated. Now, in terms of what I

84:52

would consider to be actually valuable

84:54

delegation, okay, there are two main

84:57

design patterns. The first is the

84:59

parent, researcher, and QA system, where

85:02

essentially you have a parent model,

85:04

which is usually a smart one, so this

85:05

would probably be like your Opus model,

85:07

that communicates with researchers,

85:10

plural. This will be dumber models like

85:12

Sonnet that typically do research better

85:13

and more economically. And then some QA

85:15

agents like Opus, which are basically

85:17

just tuned to QA and nothing else. And

85:21

the idea here is this is a good balance

85:22

between like those super bloated org

85:24

charts that we saw earlier, while still

85:25

allowing each type of agent to do the

85:28

things that it is inherently better than

85:29

human beings at. The parent agent is

85:31

obviously the orchestrator, anything

85:32

that is up at the top you can always

85:33

consider to be an orchestrator. And the

85:35

way you have here is you have multiple,

85:36

you know, Sonnet researchers. And this

85:38

takes advantage of that fan out idea,

85:41

okay, where when Opus needs something it

85:43

doesn't just do the research itself cuz

85:44

that'll pollute its context window. It

85:46

goes, does a bunch of research, fits in

85:48

quadrillions of tokens into the context

85:50

windows of these Sonnet agents, then

85:52

takes summaries of that, and then uses

85:54

that to make decisions. And then

85:55

basically the way that it works is, uh

85:57

and I'm just going to sort of draw like

85:58

the the the logic flow. Um Opus will

86:00

decide to do something. It delegates

86:02

down here, okay, that information comes

86:04

back to Opus. Opus then builds something

86:06

kind of on its own. After it's done

86:08

building something, it goes and gives

86:10

the uh product of its building over to

86:13

the Q&A agent. The Q&A agent returns

86:16

some changes that it suggested makes.

86:18

Opus then goes through, makes those

86:19

changes, again gives it to the QA agent,

86:22

QA agent returns. This loop continues

86:24

until basically everything is done. If

86:25

there's research that's necessary, it'll

86:27

go down, do some research here, and then

86:29

continue develop, and then finally you

86:30

have your whatever the the final product

86:32

is that you're building, whether it's

86:33

like a business system, a development

86:35

system, or whatever. In this way, you're

86:36

maximizing the incentives of each

86:37

individual agent, while also allowing uh

86:40

I want to say like the leanest possible

86:42

setup that still recognizes that

86:43

different things are better at different

86:45

types of agents are better at different

86:46

types of tasks. You know, we could make

86:48

this bigger, of course. We could have

86:49

like a testing agent. We could have

86:51

a design agent. We could have a

86:53

development agent. We could have a

86:54

back-end agent. But, you know, the more

86:55

complicated you get with the stuff,

86:56

again, as mentioned, like typically the

86:58

worst that it gets. If you want to go

86:59

even leaner than that, then the second

87:01

system is developer and QA, where you

87:03

literally just have a smart parent,

87:05

okay, and then you have a smart QA, and

87:07

then you just go back and forth between

87:08

the two. And what happens is every time

87:10

that you want to test something, you

87:11

sort of have like a Claude NMD or or or

87:14

just like a prompt that's baked into

87:15

your parent that legitimately says,

87:17

"Hey, after you're done every

87:18

development, run it through a new QA

87:20

agent." The idea here is the QA has like

87:22

literally no prompt other than, you

87:24

know, "You're a QA agent with no

87:26

context. Read this code and apply the

87:29

following whatever like design

87:31

principles to it."

87:33

And basically what occurs is this QA

87:34

agent, since it doesn't know what the

87:35

heck the project is is on, um it's not

87:38

going to be biased like the parent agent

87:39

will be in the development of the

87:41

feature. The parent agent will have

87:42

feedback from the QA agent, and so it'll

87:44

be able to incorporate it into its own

87:46

thread and take advantage of all of the

87:47

pre-existing list of failures and

87:49

successes and things it's tried and so

87:50

on and so forth. But, the QA agent is

87:52

like new, and it's new spawned every

87:53

time. And so, typically what'll work

87:55

what the way it'll work is the parent

87:56

agent will go and it'll develop a

87:58

feature, and then at the end of the

87:59

development there'll be um something in

88:01

the Claude NMD or system prompt that

88:02

says, "Okay, now that you're done, make

88:03

sure to check it with the QA agent." So,

88:05

we'll spawn a QA agent. The QA agent

88:07

will then give feedback.

88:09

Okay? The parent will design, feedback,

88:11

the parent will design, feedback, the

88:13

parent will design, no feedback because

88:15

it's now good, parent's done. And so now

88:17

we have the final product.

88:19

Um obviously, you know, because it has

88:20

to do its own research and stuff like

88:21

that, I personally think this is not as

88:23

ideal, but it is even simpler. And um

88:25

keep in mind that there is always like a

88:26

time cost every time you spin up a sub

88:27

agent. It's a fixed time cost, but uh

88:30

there's also some compound probabilities

88:32

you're multiplying because, you know,

88:33

you are having an agent delegate

88:35

something to another agent. Basically,

88:36

there's no human in the loop. The more

88:38

independent steps that an agent has to

88:40

do without a human being in the loop,

88:41

the higher the probability that it will

88:43

diverge from its sort of intended um

88:46

goal or intended task. So, when your

88:48

parent agent in the previous example

88:50

generates, you know, a bunch of research

88:52

queries to the, you know, uh

88:54

uh a Sonnet sub agents and goes and does

88:55

them, there's no guarantee that the

88:57

research the Sonnet sub agents are doing

88:58

is actually 100% faithful to what your

89:00

initial query was. Every step along the

89:02

chain that is further from you,

89:04

typically the results and the quality is

89:05

a little bit more diluted. So, yeah, I

89:07

mean, like it'd be it'd be either one of

89:08

these for me, developer Q&A or some sort

89:10

of parent researcher Q&A. That'd

89:12

basically be it though. Um personally, I

89:14

find right now with all the org charts

89:15

and stuff like that, we're just we're

89:17

just coding a little bit too much. We

89:18

definitely don't need uh I don't know

89:20

700 layers of CEOs and customer success

89:23

agents and lead engineer agents and

89:25

stuff like that.

89:27

Now, I want to talk about something

89:28

that's gotten a lot of attention

89:29

recently and does genuinely have the

89:31

potential to significantly improve many

89:33

business and programming functions. It's

89:35

called auto research. Essentially, what

89:37

I have in front of me is I have a

89:38

research lab that I've spun up to

89:39

improve the load speed of one of my

89:41

websites.

89:42

Now, the way that you gauge whether or

89:44

not a website is loading quickly is

89:46

based off of uh three main metrics. The

89:49

first is called LCP, least contentful

89:51

paint, FCP, first contentful paint. Then

89:55

there's TBT. I don't know what that

89:56

stands for. And then finally, there's

89:57

performance score. And so this is a

90:00

standardized assessment called uh the

90:02

Google Lighthouse score that you've

90:03

probably seen before. And basically, it

90:05

measures like, you know, when I type in

90:07

uh 1 second copy and I press the enter

90:08

button, how fast does literally

90:09

everything on the page load?

90:11

It also checks for very minor things

90:13

like, you know, when I when I load this

90:15

website, um does the content on the page

90:17

shift around? So, my website here,

90:19

leftclick.ai, is just one of many ones

90:21

that I own, and essentially it's just a

90:23

little bit too slow right now. And it's

90:25

slow for a variety of reasons. We got

90:26

this cool like glass isomorphism

90:28

animation on the page, you know, there's

90:29

like stuff moving around and lots of

90:31

images of my team and and so on and so

90:33

forth. So, um you know, what I've

90:35

decided to do is I've decided to

90:37

basically take all of the load off of me

90:39

to make this website faster, and then

90:41

just give it all to that fleet of AI

90:43

agents to do so instead.

90:44

Auto research is basically perfect for

90:47

use cases just like this, where we have

90:48

a very defined goal, in my case to

90:51

decrease or increase a couple of

90:52

metrics, a very defined change method,

90:54

which is how you actually make the

90:56

impact, so in my case just modifying the

90:57

website code, and then a very

90:59

standardized assessment, which in my

91:00

case is that lighthouse score.

91:02

In case you have never seen this before,

91:04

basically Andrej Karpathy, who is the uh

91:06

one of the founding members of OpenAI,

91:07

and then he also was the head of AI at

91:09

Tesla for quite a while.

91:11

Um you know, he he just was doing a

91:12

bunch of research on his own for uh one

91:14

of the models that he was running, and

91:15

he's just like, "You know, do I have to

91:17

do this stuff anymore?

91:19

I feel like I'm at the point where I

91:21

could have AI actually run most of my

91:22

research for me."

91:24

Let me make a a quick hypothesis. If I

91:26

just gave all of my changes to AI, would

91:28

it be able to do the same thing that I

91:30

do while I slept, such that when I wake

91:31

up, I'll have like a big list of

91:32

improvements? And uh it turns out, you

91:34

know, he he can. And it's not that AI

91:36

agents are like better than human beings

91:37

at determining these research changes,

91:39

but it's actually quite standardized to

91:41

do conceptually. You're basically just

91:43

like looking over a bunch of different

91:44

possible things you could do, making one

91:46

tiny change, and then just evaluating,

91:48

"Hey, did that actually improve my

91:49

score? Did that make things better?" If

91:51

so, I keep it, and I just move on to the

91:52

next thing. I go over and over and over

91:54

and over and over again until finally,

91:55

you know, you you make it hundreds of

91:57

iterations later.

91:58

So, you know, in my case, like we uh I

92:00

just re-ran the test because I want to

92:01

start this from scratch to show you guys

92:02

how this works. It's actually fairly

92:04

straightforward and what I'll do next is

92:05

I'll run you guys through the original

92:07

way that AutoFour Search works and then

92:08

how to download the repo and then set it

92:10

up on your end for whatever the use

92:11

cases that you you particularly have.

92:13

So it all started when Andrej Karpathy,

92:15

who was a researcher, he used to work at

92:18

Tesla. I think he was the head of AI at

92:19

Tesla and he was also one of the

92:20

founding members of Open AI,

92:21

asked himself, "You know, uh all this

92:24

work that I'm doing, all this research

92:25

stuff that I'm doing, is there any way

92:26

to automate it?" And he found that if

92:28

you just broke down step-by-step what it

92:29

is that he actually had to do,

92:31

it more or less always went like this.

92:34

You know, he he just had a little loop

92:35

set up where, you know, he would make a

92:39

hypothesis.

92:40

And the hypothesis would be like, "Hey,

92:42

if I change X, Y, and Z, I think my

92:43

system will run faster."

92:45

Then he'd actually execute the change.

92:48

So he'd actually go and he'd adjust X,

92:49

Y, Z. Then finally, he'd assess.

92:52

And then if the assessment was good, aka

92:54

it made an improvement, then he would

92:56

just go back to this and then make

92:57

another one. Then if the assessment was

92:58

bad, aka it failed, then he would just

93:00

get rid of it and then not change

93:02

anything and then, you know, kind of

93:03

start from scratch.

93:04

And all along the way, okay, what he

93:06

would do is he would update this little

93:08

document,

93:09

which you and I could just call like a

93:11

research log.

93:13

And you know, basically the first change

93:14

would be like, "Oh, you know, this

93:15

worked. It was great." Second change,

93:17

"Oh no, it didn't work. And here's why."

93:19

Third change, "Okay, it worked. It was

93:20

great." And eventually over time, you

93:22

end up with this massive massive log of

93:23

all the different possible things you

93:25

could do to an AI to whatever your task

93:27

is and all the things that you have

93:28

tried in the past that doesn't really

93:30

change anything.

93:31

Okay, so this is made of three files.

93:33

There's a prepare.py, which in our case

93:34

is kind of pointless. Then there's a

93:36

train.py and then a program.md. The

93:38

reason why the prepare.py is pointless

93:39

is because it's just about like AI

93:41

research specifically. It's like fixed

93:43

constants, downloading the training

93:45

data, training a BPE byte pair encoding

93:47

tokenizer, and a bunch of other stuff

93:48

that just isn't really relevant. The

93:49

stuff for us though is obviously we want

93:51

to train this and and improve this

93:53

and improve our programs. We want to

93:54

improve our websites. We want to improve

93:55

some of our business functions.

93:57

These two files here, train.py and

93:58

program.py, basically underscore how the

94:00

entire thing works.

94:01

Okay, so the super important one here is

94:03

called program.md. What you do is you

94:05

basically just tell it what you want it

94:07

to do. So, for instance, hey,

94:10

um here's what you can do as an AI

94:12

agent. Modify this file, okay? Every

94:14

time you do, I want you to print a

94:16

summary of the scores and then log it to

94:18

this file. And that's literally it. It

94:20

just goes through that loop over and

94:21

over and over and over and over again.

94:22

Then the actual train.py, in this case,

94:24

it's just like the website itself uh

94:26

sorry, the the AI model um um setup

94:29

itself with all the layers and stuff

94:30

like that. In our case, right, the

94:32

example that I was just showing you a

94:32

moment ago, that's just my website,

94:34

basically. And so basically, it's just

94:35

like it has a loop set up in its prompt.

94:38

You tell it what you can change, what

94:40

you can't change. You give it some like

94:41

sort of log file that it dumps

94:42

everything to, so you have like a big

94:44

list of changes in progress.

94:46

And then, you know, after that, you are

94:48

you're basically done, honestly. You

94:49

just fire it off and let it go. And when

94:51

you do, you know, you can make some

94:52

pretty cool changes. So, you know, I

94:54

just reran the thing, and uh we're

94:55

already seeing some pretty substantial

94:57

improvements.

94:58

Uh not all these improvements are the

94:59

same ones that I was showing you guys

95:00

before. It's this research lab. Just I'm

95:02

just resetting it over and over and over

95:03

again to see if I can find anything more

95:05

interesting.

95:06

Okay, so hopefully that's pretty

95:07

straightforward. Simplest and easiest

95:08

way to do that um is just head over to

95:10

github.com/karpathy/autoresearch.

95:13

And then what you do is you just copy

95:14

this link. Okay, so how do we actually

95:15

do this? Let's open up antigravity. I'll

95:17

click open folder. Uh I'll just make a

95:18

new one called autoresearchtest,

95:21

okay? And then I'm going to open.

95:23

And I'm going to click on clone Zoom way

95:26

in so you guys could see. I'll actually

95:27

just paste this and say clone this into

95:29

our current folder, um autoresearchtest.

95:32

Just so that it doesn't do this in my um

95:35

kind of my root folder, which it's done

95:36

a couple times.

95:38

All right, so it's going to start

95:39

saying, "Hey, I want you to clone this."

95:41

So, it's going to give it a a quick try.

95:43

It's just going to dump all the files in

95:44

here. So, now we basically have the

95:46

exact same thing that we had before,

95:47

right? We have the program.py,

95:48

prepare.py, train.py, the progress, and

95:51

you know, even like a readme that

95:52

explains everything.

95:53

So, now all we need to do, if we want to

95:55

like, I don't know, train this on a site

95:56

or something, is um well, first of all,

95:57

why don't we just make a quick site?

95:58

Hey, build me a simple one-page

96:00

portfolio site for Nick Survive. And

96:02

obviously, it doesn't know what my name

96:04

is.

96:04

So, it's now going to build a simple

96:06

one-page portfolio site.

96:08

I just wanted to do it here, so it's

96:09

going to do this inside of this file.

96:12

First, it's going to ask me some

96:13

questions.

96:15

Just uh add demo information for

96:16

everything. And my goal is uh I just

96:18

want to build a brief little website

96:20

here for us, and then I just want to run

96:21

auto research on to show you guys how

96:22

easy this is to optimize things.

96:24

Now, in our case, we're going to do

96:25

website. There are a million different

96:27

things you can apply auto research to.

96:29

Auto research goes through a quick and

96:30

easy framework, but first, I'm just

96:31

going to show you guys what you need in

96:32

order to actually set this up. All

96:33

right, now what I'm going to say is,

96:35

"Excellent, I'd like you to create a

96:37

dashboard for auto research, and then

96:40

set up the auto research framework to

96:43

optimize the Google Lighthouse page

96:45

score for index.html. I want you to run

96:48

this on a local loop, and basically,

96:50

just make index.html as fast as possible

96:52

across LCP, FCP, TBT, and then also

96:57

performance score.

96:58

>> [snorts]

96:58

>> Then, give me some sort of live

96:59

dashboard view so I could watch it um

97:01

actually work in reality."

97:04

Cool. And then I'm just going to press

97:05

enter. And basically, what it's going to

97:07

do is it's going to read through all

97:08

these files right over here, and then

97:10

it's going to use all of the information

97:11

here in order to set up the dashboard

97:13

for me. And while it's working, I just

97:15

wanted to explain a little bit about

97:16

where we are and where we're going.

97:18

The initial stage of AI coding was sort

97:20

of like vibe coding. This is like 2024,

97:22

2025 stuff, where a human being, okay,

97:25

us, prompts, then an AI writes some

97:27

code, and then a human being reviews.

97:29

So, in this way, our roles were

97:30

basically relegated to writing.

97:32

We would write the prompts, we would

97:34

make minor changes where necessary, and

97:36

in that way, we like build a website or

97:37

something.

97:38

Well, nowadays, most of us do agent

97:40

engineering, and this is sort of what

97:41

the advanced part of our course um deals

97:42

with. So, this is where instead of just

97:44

dealing with one AI, we're actually

97:46

orchestrating agents, and these agents

97:47

are doing multiple things for us all the

97:49

time and then basically like returning

97:51

the results so that we could see and

97:53

then like assess and make slight little

97:54

recommended changes. So, in this way our

97:57

role is more of the director.

97:58

But Auto Research represents sort of the

98:00

the next jump from agentic engineering

98:02

to actually full independent research.

98:05

Well, now all we do is we're no longer

98:07

like actually even directing the AI

98:08

agents. We we let them handle their own

98:10

direction. What we do is we just say,

98:11

"Hey, I have a goal and I'd like you to

98:14

achieve this goal. Here's how you can

98:15

modify X, Y, and Z and here's an

98:17

assessment." And so, in this way we set

98:19

the direction, the agent just runs

98:20

completely autonomously, and then what

98:21

we are is we're basically like a

98:23

we're like a we're like a uh

98:25

principal investigator, like a

98:26

researcher at a lab somewhere. We just

98:28

say, "Hey, you know, I want you to do X,

98:29

Y, Z." and then we just go farm it out

98:31

to a bunch of uh you know, research

98:32

assistant RA monkeys to go and and do

98:34

the experiments and so on and so forth

98:35

for us.

98:36

And so, this is along a spectrum of

98:39

decreasing human involvement. And I'm

98:41

not really sure what comes next after

98:42

independent research, but uh I do not

98:44

imagine it will require human beings in

98:46

the loop essentially at all. This is the

98:48

same sort of thing that big research

98:49

labs right now are currently using to

98:51

optimize their setup. So, Anthropic's

98:53

almost certainly doing this all day long

98:55

for Claude code to make things faster,

98:57

to make things more performant. Um you

98:59

know, OpenAI is probably doing this

99:00

behind the scenes to make Codex not only

99:02

better, but even like adjust the

99:03

architecture of the AI models and so on

99:05

and so forth. They're probably doing it

99:06

across all their web properties, right?

99:08

Anyone that's really worth any salt at

99:09

this point has probably been doing

99:10

something like what I'm showing you guys

99:11

with Auto Research for at least a little

99:12

while. It's just Auto Research is uh

99:14

Carpet's his way to democratize that and

99:16

then allow people um you know, to to do

99:18

this even with like paid providers like

99:20

Anthropic's Claude. Okay, so if I go

99:21

back here, you can see this is actually

99:22

set up the Auto Research loop and it's

99:23

actually doing the research. Um which is

99:26

not essentially what I wanted to do. I

99:27

wanted to actually see the dashboard.

99:30

So, what I'll say is, "Show me the

99:31

dashboard."

99:32

because I actually want to like watch it

99:34

work live.

99:35

And then it's just paused the

99:36

optimization loop. Now it's going to

99:38

show me said dashboard.

99:40

It's we that and then um I guess it's

99:43

going to actually show it to me now in a

99:44

second. Cool. We have it right here.

99:47

Awesome. So, here is our dashboard and

99:49

we are running multiple experiments.

99:50

Obviously, this looks a little bit

99:51

different from the dashboard I showed

99:52

you guys earlier for my left click auto

99:53

research, but that's okay. I don't want

99:55

this to look the same. I want to show

99:56

you guys you can apply this to whatever

99:57

you want. Um our very first experiment

99:59

had an FCP of 464752

100:03

and a size of 12.9. What we ended up

100:05

doing is we ended up minifying the CSS,

100:06

making a bunch of changes to the code

100:08

basically and it took it from 12.9 down

100:09

to 10, which uh technically makes our

100:12

website even faster, but in reality

100:14

doesn't actually influence things

100:15

because our scores are basically the

100:16

same, at least uh speed wise. Okay, so

100:18

this is just going to continue um

100:20

operating.

100:21

I'll just say continue.

100:23

Now, in my case what this is doing is

100:24

it's currently occupying the main

100:26

thread, right? So, this is why it's

100:27

going to be writing and making changes

100:28

and stuff like that. Um at any point

100:30

time I could say, "Hey, just go run this

100:31

in the background." Or hey, I just want

100:32

you to run this in a loop using like the

100:34

Entropic um

100:35

agent SDK or something like that. I'd

100:37

supply my API key and then it would and

100:38

then it would go. And what it's doing

100:40

now is it's actually making the changes.

100:41

I guess I should probably also like open

100:42

the website itself, huh? That'd probably

100:44

make more sense. Let me actually take a

100:45

look at uh what that looks like. Right.

100:47

So, here's here's the actual website

100:48

itself. And you can see that like for

100:49

the most part, you know, it's very basic

100:51

and simple. And but what we're doing is

100:52

we're just optimizing it. We're making

100:53

it faster and faster and faster. This

100:55

may break the website in some cases.

100:57

Sometimes some minor changes like this

100:59

do.

101:00

Um but as you can see here, we've

101:01

actually like improved it by a whole

101:02

whopping 2 milliseconds, right? We made

101:05

whatever change we did that made this a

101:06

little bit slower has now been fixed and

101:07

we're a little bit faster. Then it's

101:09

just keeping each of these. So, you

101:11

know, these things will go down very,

101:12

very, very uh slightly. They'll increase

101:14

very, very slightly. But, you know, if

101:16

you uh let it go for enough loops, then

101:17

eventually you get to the point where

101:18

you're legitimately making pretty large

101:19

improvements to the least contentful

101:21

paint. Um you know, first contentful

101:23

paint and and so on and so on and so

101:24

forth. And just know that we can discard

101:26

any runs that don't actually do

101:28

anything. So, you know, in my case my um

101:31

uh like the one requirement I had for my

101:33

left click perf auto research uh run was

101:35

that you can't visually change the

101:36

website at

101:37

So, you should take a screenshot and it

101:38

should be pixel perfect compared to the

101:40

initial one, which is why it's like not

101:41

adjusting the font or whatever. But, so

101:43

I can make more or less any other

101:44

changes aside from that and uh it's it

101:46

is doing so, which is pretty neat.

101:48

Okay, so now you're probably wondering,

101:49

"Nick, so how the hell do I actually use

101:50

auto research for my own business aside

101:52

from the demo that I just showed you?"

101:53

And like what else could I apply it to?

101:55

And my rule for auto research is that in

101:56

order for you to meaningfully make any

101:58

changes, you need to have three things.

102:01

The first is you need to have a metric

102:03

that you want to optimize for.

102:05

So, in my example, what is the metric

102:07

that I am optimizing for? Well, I'm off-

102:10

obviously optimizing for my Lighthouse

102:11

score.

102:12

And so, it's a very standardized metric.

102:14

It's really simple and it's very

102:16

objective. There's no real negotiations

102:18

about what a Lighthouse score is. Google

102:20

invented it. It is what it is. That's

102:22

what I'm looking basically to to to

102:24

assess.

102:25

The second thing that you need is you

102:27

need a way to change that metric. So,

102:31

you need a way you can influence an

102:32

outcome that modifies the metric itself.

102:35

So, if you think about it in terms of

102:36

Lighthouse page score,

102:38

the direct way to modify your Lighthouse

102:41

score is just to change your website.

102:43

And the direct way to do that is just

102:44

like alter the code a little bit.

102:46

So, in my case, not only do I have the

102:47

metric, which is the Lighthouse score, I

102:49

have a direct way I can immediately

102:50

change the metric.

102:52

And then the third thing that you need

102:53

on top of that is not only do you need

102:55

the metric and then you need a way to

102:56

change the metric, you also need a way

102:58

to assess what it is that you just did.

103:01

And so, because it's kind of like in the

103:04

name, right? This is sort of contrived

103:05

example, but like the Lighthouse score

103:07

has a Lighthouse test and the Lighthouse

103:09

test just tells you what your Lighthouse

103:10

score is. So, I have like the thing I'm

103:12

trying to improve, which is, you know,

103:14

all the metrics that I just showed you

103:15

guys. I have a way to improve it, which

103:17

is modifying the website, and then I

103:18

have a way to assess that, which is my

103:19

Lighthouse page score, uh which I can

103:21

run in a loop basically immediately

103:22

after the changes. Takes me just a few

103:24

seconds.

103:25

And so, those are the three things that

103:26

you need. If I were to formalize this,

103:28

okay, and I will because I just want

103:29

everybody to know

103:31

uh and and be able to visualize it. The

103:32

three things you need in order to do

103:34

auto research

103:35

Okay.

103:36

are number one

103:39

a metric

103:41

number two, a way to influence

103:43

or the

103:45

I don't know, change method, let's call

103:46

it, which allows you to influence the

103:48

metric. And then three, some sort of

103:50

assessment.

103:52

And with the change method and the

103:53

assessment, the most important thing, at

103:55

least in in my view, is that you can do

103:56

both of these things pretty fast. Like

103:58

if your change method takes a really

103:59

long time to do, it takes like an hour

104:00

or whatever, and then your assessment

104:02

takes another hour. If you think about

104:03

it, your your experiment will only be

104:04

able to run as fast as basically once

104:06

every 2 hours.

104:07

And that's still like light-years ahead

104:08

of like a, you know, a human

104:09

experimenter. But if you really want to

104:11

see like those crazy vertical lines in

104:12

the graph as things just get better and

104:13

better and better, sort of recursive

104:15

self-improvement, you know, you need to

104:17

have a pretty short change method. So

104:19

ideally, this would take, I don't know,

104:20

let's say like 30 seconds or so. Why am

104:22

I drawing like that? I could just do

104:23

this.

104:24

You know, maybe like 30 seconds or And

104:26

ideally, the assessment would also take

104:27

maybe 30 seconds or so as well. Because

104:29

combined, what we have here is we have a

104:30

loop that can run 60 times per hour.

104:33

Or if you multiply that out, uh what's

104:34

24 * 60? Um a lot. 1,440 times a day.

104:39

I mean, like if you could run an

104:40

experiment 1,044 times a day

104:43

you know, even if like only 2% of these

104:46

are actually good, that's like, I don't

104:48

know, about 30 changes that improve. And

104:50

if every change improves things by 1%,

104:52

what you've just done, to be clear, is

104:54

you've gone 1.1 raised to 30 1.01 raised

104:57

to 30.

104:58

Which is uh 34%

105:00

improvement per day.

105:03

At least in the first day. If you had, I

105:05

don't know, let's say 90 of these

105:07

changes be good, then this math ends up

105:09

mathing way better for you. It's 2.4x.

105:12

If you had 180 of these changes, you'd

105:14

be 6x, and so on and so forth. This is

105:16

going to go basically as high as you let

105:18

it.

105:18

Anyways, so going back to my

105:19

anti-gravity here, um just seeing a

105:21

couple of the changes, it looks like the

105:22

biggest change that it has made that has

105:24

actually and actively improved things

105:26

was this jump between 45 and 627.

105:31

So, he made some change here, content

105:33

visibility auto, remove scroll behavior

105:35

smooth, that actually significantly

105:36

improved at the uh the load speed. And

105:38

so, that's what it did here, and we've

105:39

gone from 646 at the top to a fast

105:42

contentful paint here of at the lowest

105:44

619.

105:46

It looks like the least contentful paint

105:47

did not change at all. Meaning that this

105:48

currently loads in like I think 600

105:50

milliseconds or so, which is pretty damn

105:51

good. Now, kind of a contrived example,

105:53

since I just had AI build me this

105:55

simplest website ever. But, you know,

105:56

you could see with a more complex

105:58

website, one that AI built for the most

106:00

part at least initially, and then one

106:01

that AI didn't really have a lot of time

106:03

to optimize for, and it was a lot more

106:04

complex design animations and stuff,

106:06

we've actually improved that improved

106:07

that by 20%. To give you guys some more

106:09

context, there are some people out there

106:11

that have applied this to projects that

106:12

have improved metrics by like 50%. So,

106:15

Tobi Lutke

106:17

pointed this autonomous AI research

106:20

um system

106:22

over at By the way, this is the founder

106:23

of Shopify, right? Big guy. Or CEO of

106:25

Shopify, I should say.

106:27

He ran auto research on the entire

106:29

Shopify liquid code base. Now, that's

106:31

responsible for like running more or

106:32

less everything about uh Shopify. Like

106:34

it's it's their templating liquid syntax

106:36

language thing. It's it's a lot of

106:38

freaking code. And he found that after

106:39

running this for however many times, he

106:41

had 53% faster combined parse plus

106:43

render time, which is his main metric,

106:45

61 fewer 61% fewer object allocations,

106:48

another metric.

106:50

And things are just freaking printing

106:51

for him. I mean, they you know, what's

106:52

that like? Twice as fast, essentially.

106:54

To think that you could just point this

106:56

at something and go twice as fast in

106:58

like 20, I don't know, like 30 runs or

107:00

something like that, is nuts to think

107:02

about. I don't know how long this took,

107:03

maybe it was like an evening, maybe he

107:04

went to bed, woke up the next morning,

107:06

and his freaking whole code library was

107:08

twice as fast. I don't I don't know.

107:10

Um but yeah, I mean, like the fact that

107:12

he he has done this and he can do this

107:14

is obviously very impressive to anybody

107:15

that has any sort of software that they

107:16

want to optimize.

107:18

So, what are like the practical

107:19

takeaways? Um you can optimize basically

107:21

anything you want. So, in my case,

107:22

optimizing website. How about you guys

107:24

make a SaaS app? Well, you can actually

107:25

optimize that SaaS app. You can optimize

107:27

not only the front end of the SaaS app,

107:29

you can optimize the back end. You can

107:30

say, "Hey, hey, here's your server.

107:32

Here's the whole setup. I want you to

107:34

make this load as fast as possible. I

107:35

want like the request to come in

107:36

instantly. Do whatever the heck it takes

107:38

to do it. Here's a quick little test

107:39

method, you know, we we time how long it

107:41

takes for one request to come in when

107:42

you click a button." You can just tell

107:44

it that. Even if you just gave it

107:45

literally the exact transcript that I

107:47

just gave you a moment ago, it would

107:49

probably do a pretty good job so long as

107:50

you're in the auto research framework.

107:51

You can optimize random tiny things in

107:53

your business. I mean, there are

107:54

probably some like interfaces, random

107:56

little modules, and stuff like that in

107:57

your company that like, you know, could

107:58

be way faster and way better. You can

108:00

actually optimize that. You can optimize

108:01

things like customer support queries.

108:03

You could like, I don't know, have like

108:04

a prompt, let's say, that like an AI

108:06

agent uses in order to handle customer

108:07

support. And maybe you're running some

108:09

big enterprise. Maybe you're plugged in

108:11

to a big enterprise and you have the

108:12

ability to collect this data. You can

108:13

actually just like test modifying the

108:15

prompt and then like waiting, I don't

108:16

know, like an hour and then seeing the

108:18

changes. And, you know, it's an hour,

108:19

which is kind of a loop, but it's still

108:21

24 changes a day. You could like

108:22

meaningfully modify that and move that

108:24

in the direction towards your goal.

108:26

You can do cold email. That's personally

108:27

what I'm using this for. Cold email is

108:29

kind of a special case because again,

108:31

you need a fair amount more time, but

108:32

I'm still capable of doing something

108:33

like six to 10 tests a day at like over

108:35

500 to 1,000 emails per test, which is

108:37

pretty damn good.

108:38

You can optimize a bunch of other things

108:40

as well. You can optimize like your ad

108:41

creative. You can optimize your copy.

108:43

You can optimize your conversion rate by

108:44

making minor changes to a page. You can

108:46

really have AI agents optimize whatever

108:48

the heck you want so long as you have

108:48

the volume of data necessary in order to

108:50

like construct the test.

108:52

So, hopefully I made it really clear how

108:53

all this stuff works. So, all you really

108:54

have to do is just head over to, you

108:56

know, that Karpathy auto research

108:59

that Karpathy auto research, sorry, not

109:01

that one,

109:02

um library or repo over here.

109:05

Okay? And then just copy that puppy in,

109:07

clone it inside of your repo, and then

109:08

just do away on whatever task you have.

109:11

The simplest and easiest one for you

109:12

guys to see how things work are

109:13

obviously the website ones. But yeah,

109:15

just know that like you can apply this

109:16

to more or less anything. As long as you

109:17

have those three points that I

109:18

mentioned, you need a metric to

109:20

optimize, you need a change method or a

109:22

way to influence that metric, and then

109:23

ultimately you need an assessment.

109:26

Next, I'd like to talk about automation.

109:28

Specifically, automating things on the

109:30

internet. We're going to start with HTTP

109:32

requests, then we're going to move up to

109:34

browser automation, and then finally,

109:36

we're going to round it off with

109:37

computer automation. And I'll talk about

109:38

a bunch of different platforms you could

109:40

use and ways to do more or less all of

109:41

these things.

109:43

So, HTTP requests are probably the

109:45

simplest and easiest form of, you know,

109:47

internet automation. And Cloud Code does

109:49

this natively. And in case you guys

109:51

didn't know, HTTP stands for Hypertext

109:53

Transfer Protocol.

109:55

And essentially, every time I send a

109:57

request to a website, basically every

109:58

time I try and load one, what I'm doing

110:00

is I'm sending a HTTP GET request

110:03

to the server upon which my website is

110:05

located. And then my browser will take

110:08

the response, and then mark it up and

110:10

make it look all pretty.

110:11

So, for instance, let's just like rerun

110:13

that one more time. My browser, the

110:15

client, decides it wants to access

110:17

leftclick.ai

110:19

on account of I just typed it into my

110:20

freaking page. The second I press enter,

110:23

what we're doing is we're actually

110:23

sending a request over to their server,

110:26

okay, which is located at some IP

110:27

address. And that server is configured

110:29

to automatically respond to the request

110:30

of that kind by just dumping the whole

110:32

website and giving it to you. And so,

110:34

then my browser takes that whole

110:35

website, and then it like marks it up,

110:36

and now I can see it, right?

110:38

Now, you might be wondering, what

110:39

exactly is it marking up? Well, if you

110:40

view the source of the website, which is

110:42

pretty easy to do. You can go to any

110:43

website, just right click, press view

110:45

page source, and you'll see all the

110:46

HTML.

110:47

You can see that what a website is

110:48

actually sending and receiving is not

110:50

like the pretty images and stuff like

110:52

that. It's It's usually just sending

110:53

references to those images. And um this

110:55

is actually the content of the website.

110:57

My browser just has mechanisms inside of

110:59

it that just know how to turn this into

111:01

that.

111:02

Okay, so case in point, um the

111:04

definitive AI growth partner for

111:05

fast-moving B2B companies. This didn't

111:07

just like come out of nowhere. It's not

111:08

like this is like an image. This is

111:09

actual text on a page, right? If I go

111:11

the definitive, you could see that it's

111:13

actually being represented on the kind

111:16

of code of the page that is being sent

111:18

from the server every time I make an

111:19

HTTP get request. The definitive AI

111:21

growth partner for fast-growing B2B

111:22

companies.

111:23

All right. So, why is this relevant to

111:24

us? Well, because of the first aspect

111:26

>> [clears throat]

111:27

>> of any sort of browser automation, doing

111:28

things on the internet, I should say.

111:29

Not browser automation, but like

111:31

automating network tasks.

111:33

Is this Hypertext Transfer Protocol.

111:36

Claude and other AI models now have the

111:38

ability to use web tools to basically

111:40

make HTTP requests of the kind that I

111:42

just showed you. And that allows it to

111:44

do a tremendous number of things. Not

111:46

all things, but a tremendous number of

111:48

things, if you know how to use it right.

111:50

So, the simplest and easiest way for me

111:52

to demonstrate that is you can actually

111:53

just like scrape any website you want

111:55

now with Claude or any other agent.

111:56

Hopefully, it's pretty clear and obvious

111:58

how. What we do is we just take the URL.

112:00

We go back to our agent, which in my

112:01

case is this auto research one. Then,

112:03

I'm just going to say,

112:05

"Retrieve contents of this.

112:08

Just the text."

112:10

What this is going to do next is this is

112:12

obviously going to send the HTTP request

112:14

using the web fetch tool over at

112:15

https://leftclick.ai.

112:18

And now, what it what will have gotten

112:19

back, okay, is it will have gotten back

112:22

exactly what I just showed you a moment

112:24

ago, okay, which is all of this. Now,

112:26

because I said just the text, if I go

112:28

back here, you can see that it is

112:30

extracted, sort of stripped all of the

112:32

code here, and it's returning basically

112:35

just the stuff that I can actually see.

112:37

So, what did it say? Navigation, case

112:39

studies about services, reviews let's

112:40

talk, case studies about services,

112:42

reviews let's talk, the definitive AI

112:43

growth partner for fast-growing B2B

112:44

companies. See it right over here.

112:47

You know, worked with Anthropic, Notion,

112:49

Wix, HeyGen, Vizard, Lightricks,

112:50

Durable, and so on and so on and so

112:51

forth, right?

112:53

So, I guess what I'm trying to say is

112:54

like this is a simple way that I can get

112:55

data.

112:56

And so, one of the first and most

112:57

elementary uses of, you know, any sort

113:00

of coding agent is just you can automate

113:02

website scraping really easily. So, I

113:04

could give it a simple list of tasks,

113:06

and I could say, "Hey, I want you to

113:07

scrape like 400 different websites." I

113:08

could literally just give it a big array

113:10

top to bottom. It could go out and it

113:11

could do the scraping.

113:13

Now, the issue is a lot of the time,

113:15

okay, you want to go further than just

113:16

scraping, than just reading a website.

113:19

What you want to do is you actually want

113:20

to dynamically interact with a website

113:21

and change things.

113:22

So, for instance, let's say what I'm

113:24

doing is I'm getting a big list of all

113:26

of the agencies out there, the AI

113:27

agencies like Left Click, and I want to

113:28

send them all messages.

113:30

Well, [clears throat]

113:31

you know, I could just scrape every

113:33

single website to see if there's an

113:34

email address, right? But, in my case,

113:36

maybe there's no email address. So, what

113:37

do I want to do? I want to take that

113:38

next step. The way that I do so is

113:39

usually through some sort of form or

113:41

whatever. How do I automate the clicking

113:43

of a specific button?

113:45

It's kind of difficult to do, right? I

113:46

can't just automate the clicking of a

113:47

specific button um through an HTTP

113:49

request because, you know, this is

113:52

something more than HTTP. It's kind of

113:54

JavaScript. I could try, and some

113:56

websites I'll be able to. So, hacking

113:57

this,

113:58

"Hey, uh extract the cal.com link for

114:02

me,

114:03

and then open

114:06

in Chrome." Now, going one step further,

114:08

okay, we're going to open this link in

114:10

Chrome, so we actually have this link

114:12

available. And there are some services

114:13

out there where you can actually just

114:14

send an HTTP request to actually like

114:16

book a meeting on a page. But, you might

114:18

think like in order to do that, I'm

114:18

actually have to click on this button

114:20

and then type this in and then enter a

114:21

bunch of information and so on and so

114:22

forth. Turns out I can actually just use

114:24

HTTP requests. So, I'm just going to

114:26

say, "Book a meeting for 3:30 p.m.

114:28

tomorrow. First name test, last name

114:29

test, email nick@test.com." And without

114:32

any more information, what it's going to

114:33

do

114:34

is it's going to go and it's going to

114:35

find the API documentation.

114:37

It's then going to check the

114:38

availability using the API

114:39

documentation, and then finally it's

114:41

going to ask to book. So, I'm going to

114:42

say 3:30 p.m. March 30th.

114:44

Then it's going to go and actually do

114:46

the booking. But, do you notice how many

114:48

issues there are and errors there are

114:49

with this? This obviously isn't perfect.

114:51

Now, I could theoretically figure out

114:52

the exact schema and format that I need

114:54

to use in order to send requests like

114:56

this every single time that I try and

114:58

book like a cal.com. But, the The is

115:00

like not everybody's going to have a

115:01

cal.com. What I'm doing here is I'm

115:03

building a very particular solution that

115:04

solves my one particular problem, the

115:06

HTTP request. And even then, you know,

115:08

there's just going to be some back and

115:09

forth. It's not going to be It's not

115:10

going to be perfect. And this has taken

115:11

forever. I mean, I've been sitting here

115:12

for like 10, 15 minutes. It's trying its

115:15

best. It's booking with a variety of

115:16

different means. And I don't know, who

115:19

knows? Maybe it'll actually go and do

115:20

the booking. Okay, there we go. We

115:21

actually did end up doing the booking.

115:22

Thank goodness.

115:23

That said, that took forever. It was

115:25

obviously a very fragile solution that

115:27

only works with like particular cal.com

115:28

pages, right? So, that's where we move

115:30

to the next level of automation. That's

115:32

where we go from simple HTTP request,

115:34

which, you know, most services out there

115:36

will have some sort of API application

115:37

programming interface that you can

115:38

actually communicate with.

115:40

But, um, you know, they're super

115:41

fragile. They require very particular

115:42

formats. And as you can see, they they

115:43

can take a really long time. And then

115:44

they're very narrow.

115:46

That's where we move from sort of like

115:47

the first level of automation, HTTP

115:50

request, all the way to full-scale

115:51

browser automation, which is where uh

115:54

Cloud actually fully controls your

115:55

browser.

115:56

And, you know, there there are a couple

115:57

of built-in tools with this now, but

115:59

typically the best way to do this is

116:01

using one of the two tools, at least as

116:02

of the time of this recording, um Chrome

116:05

DevTools

116:06

MCP,

116:08

or there's also the browser

116:11

use platform, which actually is pretty

116:13

new, pretty recent, but it costs a fair

116:15

amount of money. And so, what this does

116:17

is instead of just sending HTTP requests

116:18

under the hood,

116:20

what this does is it actually loads up a

116:21

whole browser for you and then goes

116:22

through the process of doing a booking.

116:24

So, you see how hard it was for me to do

116:26

this, you know, sort of simple task of

116:28

like booking a meeting on a calendar

116:29

even though I gave it the exact time,

116:31

the exact information, and so on and so

116:32

forth. That might just take a human

116:33

being 1 second. It took me like

116:35

something like 5 minutes of back and

116:36

forth and probably like 40 bucks of

116:37

tokens. So, meanwhile, I can open up a

116:39

page that has Chrome DevTools MCP, and I

116:41

could basically say, "Go here, book a

116:44

30-minute meeting for, uh, I don't know,

116:46

March 30th at 3:00 p.m.,

116:49

Nick Test, nick@test.com,

116:53

answer a bunch of demo stuff for any

116:55

booking cues."

116:57

Can I I I I just want you to look at

116:58

what's going on. I was just using Chrome

117:00

somewhere else, so it's just going to

117:01

kill the pre-existing um instance, but

117:03

now it's actually going to open up a new

117:04

one. I want you to notice that like this

117:05

is actually like opening up a freaking

117:07

instance on my browser. And then it's

117:09

scrolling through and it's clicking on

117:11

buttons and navigating on them uh

117:12

navigating through the page for me. It's

117:14

literally doing this by modifying the

117:16

JavaScript of the page and running brief

117:18

little commands in order to like

117:20

communicate and go through things. So,

117:21

it's filling up the phone number, what

117:23

made you want to contact Next Team,

117:25

what's the project budget, please share

117:27

anything that will help us prepare and

117:28

so on and so forth. I think the project

117:29

budget in this case might not actually

117:30

be um 5 or 10 I I don't even think

117:32

that's a an option cuz we don't go that

117:34

cheap. As you can see here, it's finding

117:36

the options for the budget, selecting 25

117:38

to 50K, and then it actually goes

117:39

through and it does it.

117:40

So, what are we learning from this

117:42

experience?

117:43

This is much more general, okay? It

117:46

works way better for much wider variety

117:48

of use cases.

117:50

But it's also a lot slower.

117:51

Right, this is something that previously

117:53

um I could have just sent one HTTP

117:54

request once I know the format, and then

117:56

I would have like booked that for like

117:57

0.2 seconds, right? But now, you know,

117:59

we're kind of going through the page one

118:00

step at a time. Every single one of

118:02

these actions realistically is kind of

118:03

like a it's it's almost like the same

118:05

amount of time that a single HTTP

118:06

request would take. Now, what it's doing

118:08

is actually deleting um you know, my

118:10

numbers and trying to reformulate

118:11

numbers and stuff like that in order to

118:12

like make it a valid phone number. And

118:14

you know, after a little bit of

118:15

finagling, it it actually ended up

118:16

finishing it, which is nice. So, it

118:17

actually went through, it confirmed it,

118:19

it then went through the booking process

118:21

and so on and so forth. It actually took

118:22

screenshots the whole way through of the

118:24

process.

118:25

So, um why am I showing you this now?

118:27

Because

118:29

basically this is a gradient

118:31

where it takes more setup time

118:35

to do browser or any sort of automation

118:37

via HTTP request,

118:39

but it's faster

118:41

and usually cheaper.

118:44

And there's a spectrum where we go from

118:46

more setup time, faster and cheaper, to

118:48

basically always works, but more

118:51

expensive and slower

118:53

assuming that you you don't.

118:55

And so what does that mean? That means

118:56

for any sort of like prototyping

118:58

business application on a browser, I

118:59

typically use browser automation or even

119:00

computer automation which I'll talk

119:01

about. And then once I've sorted out

119:03

that it works, I'll actually go and I'll

119:04

see, "Hey, can we do this via an HTTP

119:06

request?" because if so, it'll be way

119:07

cheaper and then we can just run a bunch

119:08

of HTTP requests in the background.

119:11

That's important because like most of

119:12

the time like the cool stuff that you

119:13

can do with cloud is actually just like

119:15

automation, right? So understanding sort

119:17

of this trade-off between pure HTTP

119:19

requests which typically function off of

119:20

like, you know, hidden APIs or whatever.

119:22

And then browser automation full

119:23

computer automation um will let you be

119:25

able to control a lot of things much

119:27

better.

119:28

So that's just one example of browser

119:29

automation. I could I could use browser

119:30

automation for anything. Hey, I'm

119:32

considering renting in Vancouver, BC

119:34

looking for $3,000 a month uh max

119:36

one-bedroom rentals somewhere in the

119:38

downtown core or in buildings that have

119:39

cool amenities like pools and stuff. And

119:41

then the bottom two are sort of like our

119:43

budget options. I could stick that puppy

119:44

in there and then it'll actually go

119:46

through and, you know, navigate to some

119:48

rentals.ca page.

119:50

I couldn't do this via HTTP requests uh

119:53

without spending a lot of time sorting

119:54

all this stuff out. And even then it

119:55

would be very fragile because the way

119:57

that these websites work is they

119:58

actually like explicitly try and go

120:00

anti-automation. They make it like

120:01

really, really difficult to do anything.

120:03

But um you know, in this case what can I

120:05

do? I can actually just open it up. I

120:06

can change a couple of filters and I can

120:07

actually go and like zoom in on the

120:09

page. It

120:10

It can do whatever the heck. It can use

120:12

the stuff on the right-hand side. I

120:13

could You could use the stuff in the

120:14

middle.

120:15

It can thumb through things. It can get

120:16

me like a big list of apartments and so

120:18

on and so forth.

120:19

And I mean like the trade-off here is

120:20

this is going to take a fair amount of

120:21

time, right? Like as you see, it's like

120:22

one action every 5 seconds or so. But

120:24

it's so general that I could just give

120:26

it a task and it will go and do it. You

120:28

know, if I were to try and do this by

120:29

saying, "Hey, go scrape the rentals.ca

120:32

web page or whatever." That that would

120:33

take so much time in order to build to

120:36

the point where it doesn't just error

120:37

out. And then most websites are also

120:39

very anti-browser anti-HTTP request

120:41

automation cuz this is the simplest and

120:42

easiest one. So you end up just getting

120:44

like error error error. This actually

120:45

like uses my browser. It's is of neat,

120:47

right?

120:49

Anyway, I'm just going to let all this

120:50

stuff go and uh in the meantime talk a

120:52

little bit about browser use, which I

120:53

think is probably like the the next

120:55

level up.

120:58

It's just called browser use the way the

120:59

uh AI uses the internet. I don't know

121:01

how long this is going to end up being

121:02

sort of like the the way to go, but

121:04

basically this is like the next level up

121:06

from Chrome DevTools MCP.

121:08

Where you give it some very simple in

121:10

instructions and stuff like that, like

121:12

fill in my loan application, and it'll

121:13

actually go through the form using

121:15

something very similar to what we did.

121:16

Maybe it uses Chrome MC Chrome DevTools

121:18

MCP in the head, I don't know.

121:19

Um and you do it for, you know, like a

121:21

bulk one-time payment of 100 bucks plus

121:24

like pay-as-you-go via credits.

121:26

So in my case, I'm not like affiliated

121:27

with this company at all, to be clear.

121:29

So I'm not going to touch on it too

121:30

much, but obviously it's a pretty cool

121:31

product. Um the big draw, I would say

121:34

for most people here, is just like HTTP

121:36

requests can be blocked because of uh

121:38

you know, platforms and stuff like that

121:40

just being scraped all the time so they

121:42

try and stop you.

121:43

Um so too can Chrome DevTools MCP be

121:45

blocked in like any sort of like instant

121:47

browsers. This platform, like basically

121:50

the whole point, you know, just to kind

121:51

of cut to the, you know, the pricing

121:53

page and all that stuff, like like 99.9%

121:55

of the reason you would want to use this

121:56

is because it is completely

121:57

undetectable. Um you could make HTTP

121:59

requests

122:01

um

122:02

sort of the old school way and then try

122:04

proxies and stuff and maybe that'll

122:05

work, but maybe it also won't. But if

122:07

you go Chrome DevTools MCP and that

122:08

doesn't work, this is what you do and

122:09

it's basically like 99.9% perfect. Um it

122:12

does this because it fingerprints, aka

122:14

like gives every one of your browser

122:16

instances uh that are just controlled by

122:18

AI like this hyper custom sort of

122:20

profile so it seems like it's like a

122:21

request that's made from a real person.

122:23

And then in that way like just like

122:24

obfuscates it all. So for most purposes

122:26

like I still use Chrome DevTools MCP and

122:28

this is like my main pick, but if I have

122:29

anything that like I need to do in sort

122:31

of a sneaky way, and uh when I say

122:33

sneaky way here, I mean like this is

122:34

great for stuff like social media.

122:37

So if you want to do like Facebook

122:39

scraping or Instagram scraping or if you

122:40

actually want to like interact with and

122:42

like posts and comments and stuff,

122:43

pretty tough to do just right out of the

122:44

box sort of with like a a virgin Chrome

122:46

DevTools MCP. But, this is really,

122:48

really good at like posting, um sending

122:51

DMs, X connect requests, what whatever

122:54

the heck you want to do.

122:55

Um so, yeah, not affiliated with that

122:57

company at all, but it is pretty sweet,

122:58

and I think that that's they're probably

122:59

going to remain the market leader in

123:00

there.

123:01

But, anyway, so just like HTTP requests

123:04

had a lot of setup time, but they were

123:05

faster and cheaper once you set them up.

123:07

Browser automation was kind of like a

123:08

good like middle ground where it's like

123:10

um oh, you know, like this actually has

123:12

some some basic browser functionality

123:13

built in, and like it's pretty obvious

123:15

how to like click a button or whatever.

123:17

Computer automation is sort of like on

123:18

the far end of the spectrum

123:20

where basically no matter what you throw

123:21

at it, it will always work. The downside

123:24

is it's very expensive, takes a

123:26

tremendous number of tokens at least

123:27

right now, and it is very, very slow.

123:29

And the way it does this is, you know,

123:31

whereas HTTP requests manipulated like

123:33

APIs and curl requests, curl's actually

123:36

lower case, browser automation

123:38

manipulates JavaScript and um I don't

123:41

know, like page clicks, like button

123:43

clicks.

123:44

Computer automation literally controls

123:46

your mouse

123:47

and your keyboard.

123:50

And because it controls your mouse and

123:51

your keyboard, you can do more or less

123:52

whatever the heck you want. Like, I

123:53

could literally like you could take my

123:54

mouse, and then I could go all the way

123:55

up here, and then I could close that

123:57

tab. Could move this all the way to the

123:58

left, it could close that tab. Like,

124:00

basically it it can do anything on the

124:01

computer that I can do.

124:03

Now, the way you do this right now is

124:04

you got to use the Claude desktop app.

124:05

So, I'm going to head over to Claude,

124:07

and then I'm going to open that up, and

124:08

then I think it's currently available in

124:09

both co-work co-work and code, but I'll

124:12

just move over to the co-work tab. And

124:13

I'll say, "Have computer use

124:17

scan through my downloads,

124:20

find the um

124:23

image called Maker School 26

124:26

or something,

124:28

and then

124:29

rename it to weekly community

124:32

call picture."

124:36

And the reason why I'm doing this is

124:37

because every name week I have a weekly

124:38

community call and then I always just

124:40

lose where the images that I use as the

124:41

thumbnail. And what it's going to do to

124:43

start this is actually going to whip up

124:45

like computer use. So, it's going to

124:46

request access to my finder and now as

124:48

you can see here it's actually whipped

124:49

up like a computer use thing.

124:51

So, now it's going to go through and

124:53

actually like type in my downloads

124:54

folder or whatever.

124:56

Navigate over there and it's just going

124:58

to start typing a bunch of different

124:59

things like Maker School and Maker

125:01

School 26 and probably try multiple

125:04

variations of like Maker School, Maker

125:06

School underscore and so on and so

125:07

forth.

125:08

Because it's using my mouse and my

125:09

keyboard, you know, it can actually like

125:10

scroll through and uh and do things.

125:13

Now, this is like local browser

125:14

automation. This is actually literally

125:16

exactly what I want, which is nice. I

125:17

could have done this in like 30 seconds,

125:18

but it's nice that it's figuring this

125:20

out.

125:21

It's using like a local browser uh

125:23

sorry, local

125:24

automation here

125:25

to like click through, scroll down, and

125:27

stuff like that. If at any point time I

125:29

want to change it, I'll say, "No, you

125:31

had it. It's the

125:34

the cover 26."

125:38

I'll press that in just so that it knows

125:39

what it's doing. All right, I just went

125:41

to grab a coffee and I got back and it

125:43

has now found the Maker School icon 26,

125:45

renamed it to exactly what I wanted, and

125:47

yeah, I guess I screwed up on the name,

125:49

but that that was what I wanted, which

125:50

is pretty cool.

125:51

So, hopefully you guys could see pretty

125:52

straightforward here to use computer

125:53

automation.

125:54

Takes a lot longer. Also consumes a lot

125:57

more tokens because it is literally like

125:58

controlling my mouse as it moves across

126:00

the page, taking screenshots of

126:02

everything as it does so, and the amount

126:04

of like fidelity that it requires in

126:06

order to do that is is pretty high.

126:07

But um yeah, I mean like eventually,

126:10

okay, put on a loop, this sort of thing

126:11

will work.

126:12

Uh it it might just take a tremendous

126:14

amount of time. Just give it a task,

126:16

say, "Keep going until you solve it,"

126:17

and it will do it. It will just probably

126:20

burn your a hole through your wallet um

126:22

while while it does so.

126:24

Realistically, the probably core play um

126:27

that I repeatedly fall on as somebody

126:29

that designs these systems for real

126:30

businesses that earn hundreds of

126:31

thousands to millions of dollars a month

126:33

is I will start with some form of

126:34

browser automation for the most part

126:36

since we're usually just doing this in

126:37

browser.

126:38

Um I'll usually try Chrome DevTools MCP

126:40

first. If that doesn't work because it's

126:42

like a stealth application or it's

126:43

something that, you know, requires

126:44

social media access, I'll do browser

126:46

use.

126:47

Once I have that flow down, you know,

126:49

unless it's like a Facebook or something

126:50

like that,

126:51

cuz uh those are just notoriously

126:53

difficult to like HTTP automate as well.

126:55

Um assuming that it's not, what I'll do

126:57

is I'll look to have Cloud Code build

126:59

like a custom utility based off of the

127:01

data that it gets from Chrome DevTools

127:02

MCP because it'll have access to network

127:04

requests and actually see the requests

127:05

that are being sent and received. Once

127:07

we have all that, then I now have like

127:08

the API internally. I write a bunch of

127:10

docs and have Cloud Code sort of like

127:12

embed that within my workspace. And then

127:14

the next time around I can just use HTTP

127:15

requests. Although, you know, keep in

127:17

mind that when you do it this way,

127:19

simply because of the volume that you're

127:20

able to hit and the fact that HTTP is

127:21

like typically a lot more regulated than

127:23

browser automation, um you know, there

127:25

are some there are some risks to that as

127:26

well. You could get rate limited, you

127:27

could get throttled, you could also get

127:28

shadow banned.

127:29

So, that's the three levels of

127:30

automating economically valuable

127:32

knowledge work through Cloud. Um it's

127:34

really just HTTP requests, browser

127:35

automation, or computer automation.

127:37

Whatever way you decide, just know that

127:38

doing that sort of automation is against

127:40

the terms of services of a lot of

127:41

platforms that you work with. So, I'm

127:43

not condoning this. I can't really

127:45

explicitly recommend it. Just making

127:46

sure that you guys understand sort of

127:48

what's available and um what other

127:49

people are doing as well.

127:52

Next up, I want to talk about Cloud Code

127:54

performance fluctuations and what to do

127:56

if and when this ends up happening.

127:59

I don't know if you guys have ever

128:00

watched that movie Interstellar, the one

128:03

with Matthew McConaughey. It's one of my

128:04

favorite movies ever.

128:06

And in it, there is a major problem that

128:08

has plagued the world that uh has, you

128:10

know, sort of set all the events in the

128:11

movie in motion.

128:13

And that's basically this idea of the

128:15

blight.

128:17

Now, what the blight is is it's some

128:20

disease that started affecting a bunch

128:22

of plants.

128:23

And as a result, something like 90% of

128:26

all of the food in the world is now just

128:27

corn. Specific type of corn. That's why

128:29

they got these big corn fields and

128:30

stuff. And then, you know, the main

128:32

character's family just does corn

128:33

farming all day.

128:35

So, in history, this idea is referred to

128:38

as monoculture

128:41

harvesting, like monoculture farming

128:43

essentially. And it's where, you know,

128:45

one particular crop is just so damn

128:48

good. It's just so freaking productive,

128:50

right? Has the highest yield and so on

128:52

and so forth. That over the generations,

128:54

the farmers learn, well, this is the

128:56

best crop ever. Why don't I just replace

128:57

all my crops with this crop? Then I can

128:59

make a bunch of crops, and then I'll

129:00

just trade this crop for other crops as

129:02

necessary.

129:04

Every time that happens, usually

129:05

productivity or yields will go up. And

129:07

they'll go up for sometimes long period

129:09

of time, sometimes like literally

129:10

generations.

129:11

And then all of a sudden what occurs is

129:13

there ends up being a problem with that

129:14

crop.

129:15

The problem is either in the soil, the

129:17

problem is maybe a bug is developed that

129:19

like really screws with that crop

129:20

specifically or something else.

129:22

And because all of the farmers' eggs

129:25

were in that one basket with that one

129:26

crop, what ends up happening is this

129:28

blight or this disease or the

129:29

circumstances of destroying all of their

129:31

crops at once.

129:33

That's led to some of the biggest

129:34

famines throughout history, I believe.

129:36

And um it's one of the reasons why, you

129:37

know, farmers nowadays do a bunch of

129:38

things, namely crop rotation. They have

129:41

multiple different crops that occupy the

129:42

same thing of land. They uh you know,

129:44

usually don't do just one crop. They

129:46

have multiple crops going, whatever

129:48

types of crops they are. Just so that if

129:50

a harvest on, you know, one type fails,

129:52

then, you know, they'll at least get

129:53

something from something else.

129:55

Well, the reason why I'm bringing up

129:56

this analogy, and I think I've really

129:57

hammered it home here,

129:58

uh is because I think this applies to

130:01

Claude Code.

130:02

Claude Code's really good. I don't think

130:04

there is a better coding harness um out

130:06

there. I don't think there really is

130:07

anything better than Claude Code, at

130:09

least as of the time of this recording.

130:10

And I don't know if there ever will be.

130:11

This is me just being honest with you

130:12

guys.

130:13

I think at a certain point with AI, you

130:14

know, an agent's ability to program the

130:16

next model, okay, just gets better and

130:19

better and better. And so, the people

130:20

that have the better agents, if if they

130:22

apply their resources effectively just

130:23

end up with like this impossible

130:24

advantage due to exponential growth.

130:26

So what that logically means is that you

130:29

know, it's the best crop ever, right? It

130:31

gives you the biggest yields ever

130:33

because it's so productive because it

130:34

makes you productive. You're probably

130:35

just going to use it all the time.

130:37

The downside to that is there are a lot

130:39

of things here outside of our control in

130:41

terms of Claude code performance and

130:42

sometimes Claude code performance goes

130:44

up and it goes down and other times it's

130:45

just completely gone.

130:47

So the reality is we're probably all

130:49

going to be using Claude code a lot

130:50

because Claude code as mentioned is

130:52

freaking awesome. But if you grow too

130:54

rely on it to the point where Claude

130:56

code is basically a monoculture crop,

130:58

you end up with situations like this

131:00

which actually just happened yesterday.

131:02

Just one of many occurrences.

131:04

To make a long story short, Claude went

131:05

down. You know, there was a big issue

131:07

with Opus 4.6 and I think it lasted like

131:10

maybe an hour or so.

131:11

And basically 95% of developer

131:13

productivity

131:15

plummeted the second that Claude was

131:16

gone.

131:17

The reason [snorts] why is because, you

131:18

know, Claude was everything. They stored

131:20

all their files on, you know, the Claude

131:21

desktop app with

131:23

simple skills that were just made in

131:25

like Claude's format and

131:27

nobody or nothing else's.

131:29

The second that Claude,

131:31

you know, was down then all their

131:32

prompts that they had saved in specific

131:33

points and stuff like that were very

131:35

difficult to access and they weren't

131:36

good to use with other models. Whole

131:38

code bases that have been designed by

131:39

Claude were not interpretable at all.

131:41

There were no comments so they tried

131:42

using other models and other agents and

131:44

like that didn't really work. And then

131:45

ultimately Claude is just the best. The

131:46

intelligence of the these other agents

131:48

just don't work the same. So you know,

131:50

it just led to like a bunch of bunch of

131:51

issues essentially.

131:53

This isn't the first time that this has

131:54

happened. This has actually happened a

131:55

number of times.

131:56

You know, this is Adam from earlier

131:58

today talking about like major outages

131:59

with Claude and how different types of

132:01

platforms are operational whereas other

132:03

ones aren't. There's also a bunch of

132:06

Claude code performance degradations.

132:07

You know, I just looked up an old post

132:09

from I think it was Theric here who's

132:12

one of the lead guys on Claude code. He

132:14

like drops Claude code updates and stuff

132:16

all the time.

132:17

Well, anyway, um you know,

132:19

there were degradations historically

132:21

since December 17th, 2025 of Opus 4.5

132:24

and Claude code, where basically because

132:26

of some runaway other garbage collection

132:29

or some sort of like memory issue,

132:31

um you know, Opus just got worse and

132:32

worse and worse and worse every day for

132:34

a certain period of time, which led to

132:35

like, you know, massive performance

132:36

decreases,

132:38

um literally probably on planet Earth,

132:39

at least in knowledge work.

132:41

So, okay, hopefully at least at this

132:42

point I've convinced you guys why Claude

132:45

is nowadays probably already pretty

132:47

monocultury and likely as it continues

132:49

to to dominate, it's likely to just

132:51

become more and more and more

132:53

monocultury over time.

132:54

Um the question, obviously, is what the

132:56

hell can we do about it?

132:57

And so, there are a couple of uh

132:59

solutions, and most of them revolve

133:00

around this idea of diversification,

133:02

where basically, you know, instead of

133:03

just putting all of your eggs in the

133:06

Claude basket, this is my cute little

133:07

basket, sticking it chock-full of, you

133:10

know, nice Claude eggs,

133:12

what we do is instead of putting, I

133:13

don't know, all 10 of our product theory

133:15

eggs in this Claude basket, we put like

133:17

seven, eight, or maybe nine in them,

133:19

okay? So, maybe like seven out of 10 in

133:22

Claude.

133:23

And then, what you do with your other

133:24

three out of 10 is you just distribute

133:26

them. You distribute them such that, you

133:28

know, I don't know, one out of the 10

133:30

are in Codex.

133:33

You know, another one out of 10,

133:35

my god, I'm going to get really good at

133:36

drawing these, are in uh I don't know,

133:38

like anti-gravity's like Gemini, right?

133:42

And maybe

133:43

one out of 10

133:44

are in some other type of coding

133:46

harness,

133:48

like a Pi or something, that maybe also

133:51

uses like some form of like local models

133:53

or whatever.

133:54

The point that I'm making is, obviously,

133:55

we're being pragmatic here. Like, you

133:57

should probably predominantly use the

133:58

best model out there because, you know,

133:59

it's not like a it's not a linear thing.

134:01

If a model is like 1% better than

134:03

another model, that 1% once you get

134:04

smart enough is like the difference of

134:06

like a gulf, right? Einstein is like 1%

134:09

smarter than a

134:11

normal human being or something like

134:12

that, and he was able to come up with

134:14

the theory of relativity or something

134:15

along those lines. Obviously, don't take

134:17

me at face value there. I'm sure his IQ

134:19

is through the roof. But the point that

134:20

I'm making is like when you get to this

134:21

point with these weird galactic

134:22

intelligence, even like a small little

134:24

increase in the intelligence the model

134:26

may lead to like a big downsize

134:27

difference, right? So, if you have the

134:29

ability to use the best model, just use

134:30

the best model. But don't put all your

134:32

eggs in that basket because if that

134:34

occurs, then what will basically happen

134:36

is like as the performance of Claude

134:41

over time goes up assuming I'm Claude is

134:43

orange,

134:46

your total productivity

134:47

in blue here will also go up basically

134:50

in lockstep. And so if the performance

134:52

of Claude goes down, so too is your

134:54

entire productivity. If the performance

134:55

of Claude goes up, so too does your

134:56

entire productivity.

134:58

Instead, diversify, okay? Instead of

135:00

just this like yellow one which is

135:02

Claude, maybe you have like a green one

135:03

here which is Codex. And what occurs is,

135:05

you know, Codex maybe is a little bit

135:06

more like this.

135:08

And so what ends up happening is the

135:09

average performances of, you know, both

135:11

of these sort of average out and then

135:13

instead of being super reliant on

135:15

Claude, what you get to do is, you know,

135:16

this black thing which is like you ends

135:18

up being a lot more stable. It's the

135:20

same thing in investing. Have you guys

135:22

ever invested in, like, I don't know, an

135:24

ETF or some sort of

135:28

index fund? You know, basically the way

135:29

that all of the stocks work is there'll

135:31

be a stock that does this, there'll be

135:33

another stock that does that.

135:35

There'll be another stock that does

135:36

this.

135:37

There'll be another stock that does

135:38

this. Do you see how volatile? Okay,

135:40

that stock probably isn't going back. Do

135:41

you see how all volatile all these

135:43

different stocks are? Well, rather than

135:45

tie your your literal life savings to

135:47

all, you know, any one of these stocks,

135:48

you just tie them to all of them

135:49

simultaneously. Such that, you know,

135:51

over time maybe your thing slowly goes

135:53

up and that's a lot more reliable and

135:54

dependable.

135:56

Okay, so the way that you do this in

135:57

practice, the way that you diversify

135:59

your models in practice,

136:01

um, is you use platforms built in that

136:03

have the ability to orchestrate or

136:05

juggle multiple different types of

136:07

agents just inherently.

136:08

Or you use things like MCP servers or

136:10

whatever that allow you to do that sort

136:12

of thing within Claude code or within,

136:13

you know, some other um you know, coding

136:15

agent. And so obviously like right now,

136:17

okay, if I'm just being pragmatic with

136:19

you, there's there's Claude code and

136:20

that's sort of like the big boy.

136:22

And he's they're fantastic. Then

136:24

there's, you know, Codex. And some

136:25

people will swear uh on their mother's

136:27

life that Codex is way better than

136:28

Claude code, but I I don't really think

136:30

so.

136:31

And then there's like, you know,

136:32

um Gemini isn't really that the right

136:34

term. It's sort of like Antigravity's

136:36

like agent chat

136:38

within um

136:40

um uh Antigravity.

136:42

Okay, and this is sort of like my little

136:43

personal tier list. But basically, you

136:45

know,

136:46

use other models in conjunction with

136:48

harnesses and stuff like that you might

136:49

have set up in Claude code for for best

136:51

results.

136:52

Okay, so yeah, anyway, there are two

136:53

main major ways of doing this right now.

136:55

The first is using a platform like

136:56

Conductor. If you've never seen a

136:58

platform like Conductor, what this does

137:00

right now is allows you to create a

137:01

bunch of parallel Claude X and Claude

137:03

code agents inside of isolated

137:05

workspaces on your computer. You can

137:07

then just like with Antigravity or, you

137:09

know, Claude code desktop app order, you

137:10

can just see how their performances and

137:13

what they're doing sort of in real time.

137:15

And because you are just the conductor

137:17

up at the top, if, you know, the Claude

137:19

code chunk of these don't end up

137:21

working, but then the Codex ones do,

137:23

then that's perfectly fine. It doesn't

137:24

really change anything for you. You're

137:25

just going to like momentarily allocate

137:26

most of your time and energy to the

137:28

Codex ones. It's on the exact same

137:30

interface. It's very straightforward.

137:31

You just do it all, you know, through

137:32

this sort of like conductor interface.

137:34

Super easy. And then uh you know, like

137:36

this is used by a lot of real big people

137:38

all over the place to basically average

137:39

out minor statistical fluctuations in

137:41

models and then allow

137:43

for the taking advantage of different

137:46

parts of different models that are

137:47

slightly better, slightly worse than

137:49

each other at things. Like for instance,

137:50

a lot of people think that Codex is

137:52

actually like quite cracked at, you

137:53

know, the sort of like deep

137:54

contemplation required to make big

137:55

backends and it's better than Claude

137:57

code. I don't know if I entirely agree

137:58

with that and I think even if that were

138:00

correct today, it probably would not be

138:01

correct in like a few weeks because

138:02

things change so quickly. But, you know,

138:04

this allows them to take advantage of

138:05

Codex's ability to build the most

138:06

cracked back end ever and then have a

138:08

cloud code do some other thing that

138:09

cloud code is great at.

138:11

Okay, so conductor's pretty sweet. Uh

138:12

I'm not going to worry too much about

138:14

like setting it all up. It's actually

138:15

quite self-explanatory.

138:17

And I don't want to just make like a

138:18

700-hour YouTube video that's me,

138:20

you know, setting up a bunch of

138:21

different platforms. There's no real

138:22

value to this. These guys set out the

138:24

knowledge the documentation really,

138:26

really plainly and really intelligently

138:27

done here. You can just click that

138:28

download button, set it up, and then

138:30

you'll be good to go.

138:31

Okay, so that's number one, right?

138:33

Number two is you can use something like

138:36

um MCP servers to distribute your load

138:39

across multiple different models.

138:41

So, for instance, there's this Codex MCP

138:43

server, which, you know, technically

138:45

lives in cloud code. So, if cloud code

138:46

does go down or something like that, you

138:48

won't necessarily be able to use it.

138:49

Keep that in mind. Um but, you know, if

138:51

it's just one of the cloud models or

138:52

whatever, it's a little bit different.

138:53

Basically, what you do is you download

138:55

an MCP server that allows you to

138:56

communicate back and forth with a Codex.

138:59

And so, that one's very straightforward

139:00

and easy. There's There's a get

139:02

repository right over here. It's very

139:03

straightforward. All you do is you

139:05

literally just like install the Codex

139:06

CLI, okay, using npm i -g @open

139:10

ai/codex. Then you just give it your

139:11

open AI API key. Then you just add it to

139:13

cloud code, then you can actually just

139:14

like have a conversation with them. So,

139:16

for simplicity's sake, I'm actually just

139:17

going to do that cuz that's a lot

139:18

faster. I'm just going to go back to my

139:20

anti-gravity instance, which is right

139:21

over here. You can see I got a search

139:23

back a little while ago from something I

139:25

was working on. I'm just going to open

139:26

this up and I'll say install this.

139:30

I'll say keys in .env, don't share, this

139:34

is a demo.

139:36

Let me know when done so I can restart.

139:39

And what it'll go through is it'll go

139:40

and install the Codex MCP server. And

139:43

then I can just go here and I could say,

139:44

"Hey, ask Codex how it's going."

139:48

So, now what it's going to do is rather

139:50

than just, you know, kind of operating

139:51

on its own thread, it'll literally just

139:53

run through like a a thing pinging Codex

139:55

and saying, "Hey, man, what's going on?"

139:57

It echoed back the message successfully.

140:00

Okay, I want to chat with Codex. Yes.

140:03

And let's just hear uh what it has to do

140:05

what it has to say, rather.

140:07

So, Codex-CLI Codex, this is just a

140:09

ping, I guess, to make sure that it's

140:10

online. This one is now saying, "Hey,

140:12

I'm running on Codex uh on GPT-5 in your

140:15

local coding workspace. I can do all

140:16

this stuff. The file system's currently

140:18

restricted and so on and so forth."

140:19

So, I mean, this will work in the cases

140:21

where you want Claude to like

140:23

orchestrate a conversation with Codex.

140:25

So, that actually I'm going to go into

140:26

Codex. And that can that can be quite

140:27

good when um you know, you don't really

140:29

want to like upset your local workflow.

140:30

You still want to work within Claude

140:31

code and do everything that you're

140:32

normally doing. But then for whatever

140:34

reason Claude code performance has been

140:35

degradated. Degradated.

140:37

Degradated? Degraded. Uh but I should

140:39

note that, you know, if Claude code

140:41

itself goes down, like say there is some

140:43

widespread Anthropic outage, you know,

140:44

your your next best bet is to literally

140:46

go and download probably like the Codex

140:49

um desktop app here. Download it for

140:52

macOS. And either get a subscription or

140:55

at least know how to get a subscription,

140:56

know how to use the app such that if

140:58

there are major issues with any one of

140:59

these platforms, you know, at any point

141:01

in time you can just jump right back.

141:02

So, that's personally what I do. I

141:03

actually have Codex up and running. I

141:05

know how to use Codex. I'm very familiar

141:06

with Codex. Um you know, the way that I

141:08

set up my workflow is not only do I have

141:11

like a dot Claude with the skills and

141:13

and you know, so on and so forth. But at

141:15

any point in time I just I I can just

141:17

duplicate this whole workspace such that

141:18

it's like generally accessible by any

141:20

agent. I can actually go over here and

141:22

then say, "Hey, uh for whatever reason

141:24

Claude code is down, so I'd like you to

141:26

duplicate this whole business workspace,

141:28

change anything that is Claude-specific,

141:30

like the dot Claude, the Claude.md, etc.

141:34

to um the usual agent specification. You

141:37

can find all that at agents.md."

141:40

Um and in general, just make sure all of

141:42

this stuff works for Codex. Then what

141:43

you can do is you can either run some

141:45

sort of like synchronization flow or you

141:46

could just like manually do this every

141:48

now and then. And then you can send that

141:49

off to Codex however necessary. Cool.

141:51

And now it's actually going through this

141:52

process of syncing the workspace to the

141:55

exact same type of folder {slash}

141:57

business-codex.

141:58

Then it's just changing my agents.md and

142:00

stuff.

142:01

Um what you could also do is inside of

142:02

the same workspace, you could just like

142:04

duplicate this, make this like {dot}

142:06

agents or whatever. You could have this

142:07

just all go cap agents. You just

142:09

probably need some line in your

142:11

claude.md that says, "Hey, when you

142:12

update your claude.md, also update your

142:13

agents.md or whatever." The whole

142:14

purpose of this workspace is to work

142:16

with anything.

142:17

Um in my case, you know, I this is just

142:18

very Claude specific and I mean courses

142:20

on Claude so I can't really just mess

142:22

this up and I don't want the workspace

142:23

to get any any messier than it already

142:25

is. But hopefully you guys see how easy

142:26

it would be realistically to do some

142:28

form of diversification.

142:30

Okay, so just to make it super clear,

142:32

there were three main forms that I was

142:33

recommending here, right? The first form

142:35

was I recommend downloading and then

142:37

installing a tool like conductor.

142:40

What conductor does is allows you to run

142:42

a team of different coding agents right

142:43

out of the bat using like the native CLI

142:46

for codex and um Claude code. And so

142:48

you're actually having multiple agents

142:50

just like operating in parallel. They're

142:51

just doing so sort of in one workspace

142:54

that is not like branded or tied to any

142:55

individual type of model provider.

142:58

The second one is using something like

142:59

the codex MCP server, which is great to

143:01

use when like Claude code is up but

143:03

individual Claude models are degraded or

143:05

there's some issue that are that is

143:07

preventing it from operating the way

143:08

that you want it to.

143:10

In that way you could still take

143:10

advantage of whatever Claude model you

143:12

do have access to and also like your own

143:14

Claude interface, let's say in Claude

143:16

code's desktop app or maybe like an

143:18

anti-gravity um Claude code extension

143:20

setup like I have.

143:22

And then the third is just operating in

143:24

an entirely different agent platform

143:25

entirely. Um my recommendation at least

143:27

as of right now is to use codex because

143:29

every test that I've ran with Gemini is

143:31

nowhere near as good um at anything

143:33

except for front-end design. Perhaps

143:34

their new model will come out and

143:36

that'll be way better or something like

143:37

that, but I'm not going to hold my

143:38

breath for that at the moment because as

143:40

mentioned, I think Claude is really just

143:41

the dominant uh the the dominant playboy

143:43

as of right now. Okay, and all of this

143:45

is because we do not want the amount of

143:47

culture crop. We do not want all of our

143:49

eggs in one basket. We can have most of

143:51

our eggs in the cloud basket for sure,

143:53

but if you put all of them in then

143:54

you're going to suffer the exact same

143:56

situation this present guy did,

143:58

where you know, the second the cloud

143:59

went down he just couldn't do anything.

144:00

Okay? So, hopefully that makes sense. I

144:02

personally have about 70% cloud code and

144:05

maybe 30% spread across codex and then

144:07

like a couple of open source models.

144:09

And then I use agnostic, you know,

144:11

coding harnesses like pie in conjunction

144:14

with things like conductor in order to

144:15

make sure that I'm good to go.

144:18

All right, now let's chat workspace

144:19

organization. I'm going to show you guys

144:21

the way that I personally organize my

144:22

workspaces, discuss a couple of

144:24

alternative ways, and then also just

144:25

talk about like the hierarchy of

144:27

information and then how to maintain

144:28

like a really root clean file space. So,

144:31

this is the structure that I basically

144:33

have set up. And I'm going to run you

144:35

guys through my actual anti-gravity

144:36

setup in a second. I actually just had

144:38

AI generate me a bunch of backgrounds

144:39

for this, so that's pretty meta.

144:41

But um to make a long story short, I

144:43

store all of my business stuff in a

144:45

business workspace. Okay, now my

144:47

business workspace includes a bunch of

144:49

additional folders that you don't really

144:50

need in order to have my structure

144:52

there.

144:53

Very specific to the platforms that I

144:55

use and and whatnot. Really, the folders

144:57

that you need, if I just cross out all

144:59

the stuff that you probably don't

145:00

actually need, okay? And like you

145:03

probably don't need this either. Some

145:04

people have virtual environments, some

145:06

don't. But really, the stuff that you

145:07

actually do need is going to be like a

145:10

dot cloud, which is where you're going

145:11

to store all of your, you know, cloud

145:13

specific files. So, it's where you're

145:14

going to store your skills, it's where

145:16

you're going to store your agents, and

145:18

ETC.

145:19

An active or a temporary folder, or

145:21

whatever the heck you want to call it,

145:22

but this is basically just going to

145:23

store everything else. So, all the

145:26

generated files and so on and so forth.

145:29

A dot ENV, where you're going to put

145:31

your obviously ENV type keys. So, any

145:34

sort of like API keys, credentials,

145:35

anything like that. And then finally,

145:37

your local cloud.md, which is just like

145:41

your local system prompt.

145:43

And if you guys remember, we stored the

145:44

global system prompts um in a

145:47

kind of like a tilde {slash} dot Claude

145:51

folder um where, you know, the rest of

145:52

your your global stuff is. And this is

145:55

like this is somewhere else. This is

145:56

usually like your home folder, wherever

145:58

that is. On a Mac, you know, in my case

145:59

it's like Nick Survive. So, if I go to

146:01

my Nick Survive folder and then I show

146:02

hidden, I can actually see the dot

146:04

Claude folder. I can click on it and

146:05

then I can see it.

146:06

On your workspace, if it's like a

146:07

Windows or whatever, it's it's going to

146:08

be different. So, you're going to have

146:09

to look for it.

146:10

Okay, so mine obviously looks a little

146:12

bit different from that, but I just want

146:13

you to keep in mind those um

146:14

you know, the dot Claude, the active,

146:16

the dot ENV, and then the Claude MD.

146:18

That sort of structure that I showed you

146:19

a moment ago. That's the one that I'm

146:20

going to be assuming that you you're

146:21

going to be building.

146:23

Okay, so I separate things into and I

146:25

also have a personal version of this,

146:27

but for now we're just going to stick

146:28

with business. A business workspace. And

146:31

so, I literally have like a folder on my

146:33

computer, you know, Nick Survive.

146:36

>> [sighs]

146:36

>> And then it goes {slash} business.

146:39

And it's within this business folder

146:41

that I currently exist, that I do all of

146:43

my work.

146:44

So, what do you have inside of business?

146:45

You have your dot ENV.

146:47

You have your Claude skills, which is

146:49

sort of like the intellectual capital

146:51

that you accumulate over time as you do

146:52

various SOP-able things. You have your

146:55

Claude.MD. Then you also have um you

146:58

know, like your active folder.

146:59

And the way that I personally organize

147:01

this as somebody that not only um uses

147:03

Claude Code and other agents in my

147:05

day-to-day life, but also sells clients

147:07

on the implementation of these sorts of

147:08

things. And then is also responsible for

147:10

using Claude Code in order to fulfill

147:11

the implementation.

147:13

Is I separate it such that my main

147:15

business needs, that contain all of like

147:18

like my stuff,

147:20

is in this business folder. And then

147:22

anything that I do on behalf of my

147:24

clients lives in specific client

147:26

folders. So, let's say I have a client

147:27

called client A. Well, client A actually

147:29

has his own dot ENV with the client's

147:31

API keys. They have a dot Claude {slash}

147:34

skills with the project skills, skills

147:35

that are highly specific to the needs of

147:37

that particular project. You know, if I

147:39

work with like some sort of digital

147:40

marketing agency and I have a skill that

147:42

I use on their behalf in order to like

147:45

connect to some service that they use to

147:47

print out a report. Like I would put

147:49

that skill inside of the client folder.

147:51

Then I also have a cloud.md. And that

147:53

essentially, you know, I just run with a

147:55

slash in it and that also just describes

147:57

a little bit about the client. In the

147:58

same way that I showed you guys earlier,

147:59

I have my own cloud.md that describes a

148:01

bunch of stuff about me. So, oh, who am

148:03

I? Nick Suriya, if you know, I'm 30

148:04

years old. I'm an n-j. I currently live

148:07

in XYZ area. Here are all my businesses.

148:09

Here's how much money I make. Here's all

148:10

this like highly relevant contextual

148:11

information.

148:12

I also have similar contextual

148:14

information for my clients and then for

148:16

their businesses as well as anybody on

148:17

their team. So, that you know, if I say,

148:19

"Hey, send a message over to Jane

148:20

letting her know XYZ." It's literally

148:22

just like one message and then and then

148:23

it's sent.

148:24

Okay, so I duplicate that across all my

148:26

client base. So, client A, client B,

148:27

client C, however many clients you have,

148:29

that's how many project folders I'd

148:30

have. And the key here and the reason

148:32

why I think this is like this most solid

148:33

organizational scheme I've stumbled on

148:35

after several years of working with this

148:36

stuff is you can actually call client

148:38

skills while still being in the business

148:40

workspace.

148:41

You know, it's not the exact same

148:43

because you're not technically loading

148:44

them inside of the If I just go slash

148:46

context here, you're not technically

148:48

loading them inside of the actual

148:50

context, okay? You only got the ones

148:51

that are like sort of local here.

148:53

>> [gasps]

148:53

>> But you can still call skills that are

148:55

not local simply by putting in your

148:57

cloud.md a one-line thing that says,

148:59

"Hey,

149:00

there's some skills that we reference

149:02

that aren't all going to live inside of

149:03

the dot cloud/skills folder. These are

149:05

client-specific skills. If you want to

149:06

reference those, then you actually have

149:08

to go inside of the client folder that

149:09

I'm referencing and then, you know, pull

149:11

it out that way."

149:12

And so in my case, you know, the

149:13

business

149:15

workspace is sort of like top level and

149:17

the client workspace is sort of

149:18

underneath. So, what's up with this

149:19

don't pollute root always store in

149:21

active or subdirectories? You know,

149:22

earlier I said I have an active folder.

149:25

The reason why is because if you start

149:26

polluting your root, it just ends up

149:28

being like a total

149:30

nuclear bomb waiting to happen. You just

149:32

have so many files. Your files are

149:33

stored all across one giant folder. Not

149:36

only is it like visually insane to look

149:38

at because it's like this is always open

149:40

essentially and it just pushes all the

149:41

way down to the bottom. Um but it's also

149:43

a little disorganized for your agent as

149:44

well. Better instead to store specific

149:47

locations that you dump files to, okay,

149:50

using the skill spec itself. So, for

149:53

instance, inside of model chat, if I go

149:55

over to my skill, you'll see that it

149:57

actually specifies where to put the

150:00

actual model chat. It literally says

150:02

dump it inside of active/model-chat

150:04

and then name it in this particular way.

150:07

So, in that way this model-chat skill is

150:09

actually hooked up over here to this

150:11

model-chat,

150:12

um you know, conversation thread. I can

150:14

open that up and I can actually like see

150:15

the conversations that we have been

150:17

having.

150:18

Um it's also much more organized for the

150:20

skill because I'm not just dumping

150:21

everything in the same place.

150:22

Uh it's super easy to do and then I

150:24

don't actually have to do any sort of

150:25

like agentic search or agentic lookup,

150:27

which I think is pretty valuable uh

150:29

because agentic lookups are just more

150:30

things that consume tokens. So, what I'm

150:32

trying to say is I just store everything

150:33

inside of like a folder I can toggle

150:35

called /active and then I store any

150:37

specific information as to where these

150:39

things will go um inside of the actual

150:41

skill themselves. So, you know, there's

150:43

a bunch of leads of my own CRM, that's

150:45

where they live. There's like some

150:46

config files for other things, this is

150:48

where they'd live. If I do research,

150:49

this is where they'd live and so on and

150:50

so forth. I would never store random

150:52

scripts directly in root. Neither would

150:54

I do temp files or data files. If you

150:56

want like temp files, files that you

150:57

know are only going to be used for like

150:58

a short period of time or in the course

151:00

of a a process being executed.

151:02

Personally, I actually store these as

151:04

like active/ .tmp inside of some hidden

151:07

temp folder so they don't even mess up

151:08

my active.

151:10

And you're probably thinking like,

151:11

"Well, won't I lose stuff if

151:12

everything's super nested?" No, you you

151:13

won't lose anything nowadays. You're

151:15

trading off the amount of time it would

151:16

take you to like scroll through your

151:17

root thing for the amount of time it

151:20

would just take you to pump it into your

151:21

agent to ask it, "Hey, can you find

151:22

XYZ?" But you'll find that if you just

151:24

like allow the agent to organize your

151:25

workspace, it it tends to do so in a

151:27

pretty consistent and then reliable way.

151:29

So long as you expressly give them a

151:31

structure where you're like, "Hey, make

151:32

sure I always put stuff in active."

151:34

And remember earlier I talked about

151:35

diversifying away from just Claude code.

151:37

Well, what's really cool is um you know,

151:39

when you run a business workspace like

151:41

this and then you have your client and

151:43

and and so on and so forth workspace

151:44

sort of underneath it, uh what you can

151:46

really easily do is just duplicate your

151:47

Claude.md into an agents and then a

151:49

Gemini.md. You can just have all of

151:51

these in all of your workspaces

151:52

simultaneously. Such that if at any

151:54

point in time you want to use, I don't

151:55

know, cursor for something, you want to

151:57

open it in anti-gravity, you want to do

151:59

it directly in Claude code. Like you

152:01

never really run out of the system

152:02

prompt um um design pattern. Like you

152:04

know, if you have the same thing written

152:05

in Claude.md, the same thing in

152:06

agents.md, the same thing in Gemini.md,

152:08

you can basically just like have that on

152:09

24/7.

152:10

Now, I haven't needed to do that

152:11

personally um in quite a while and I've

152:13

actually been very lucky to have not

152:15

been affected by some of the recent

152:16

outages. But I remember back uh I don't

152:18

know, like a month and a half ago or

152:19

whatever, I actually had like a specific

152:21

line that said, "Hey, I want you to

152:22

synchronize the Claude.md with the

152:23

agents.md and the Gemini.md all the time

152:25

just in case, you know, we have an

152:26

outage and I need to drop this into a

152:28

different coding platform." Now, another

152:29

thing that'll happen reasonably often

152:31

is, you know, because we're not dumping

152:32

stuff into our root, we're going to end

152:34

up dumping a lot of stuff into um

152:36

active, right? And so I have like just a

152:38

bunch of stuff here. Dub video links, CA

152:40

dentist, auto research, Hindi source,

152:42

you know, when I was dubbing my stuff,

152:44

bunch of different um um screenshots and

152:46

stuff like that. Uh you want to

152:48

periodically clean up this workspace. So

152:50

you periodically want to say something

152:51

along the lines of, "Hey, clean up my

152:53

active {slash} folder. Anything inside

152:56

of subfolders are fine, but anything

152:57

that's just loosely in the in the

152:59

folder, like any TXT files, PY files,

153:02

JPEGs and related, I want you to clean

153:04

up by either deciding if it's necessary.

153:06

If it's just a temp file, just get rid

153:07

of it. Otherwise, store it in a folder

153:09

that makes sense." You're going to want

153:10

to run something like this reasonably

153:12

often. Um the reason why is because you

153:13

just don't want to have to, you know,

153:15

scroll again through like a quadrillion

153:16

different things. You also want to make

153:17

sure that any future model that comes

153:19

around can just like very logically look

153:20

at uh uh some sort of organizational

153:22

hierarchy and then make decisions based

153:24

off of that. So, that's what's going on

153:26

here with all these docs for I closed,

153:28

right? It's deciding what to do here.

153:30

It's going to download them into

153:31

different folders. It's actually going

153:32

to get rid of a couple files here like,

153:34

"Hey, this is a file. This is an

153:35

incomplete download. This is a bunch of

153:37

unnamed temp snapshots, right?" And And

153:39

what you'll find is within like 2

153:40

seconds it just does the whole thing.

153:41

So, now my active folder is much much

153:43

cleaner and I don't have to worry about

153:44

this sort of thing ever again, which is

153:46

nice.

153:47

Um and you know, in my case I also have

153:48

a couple of these web design projects.

153:50

Enumerate web design in active. These

153:53

are things like Volta,

153:55

Aura, and so on and so forth. Find

153:57

similar projects and then store all of

153:59

them within a web dash design folder.

154:01

And despite the fact that, you know, you

154:02

might be like thinking, "Nick, why the

154:04

hell are you spending time and energy

154:05

doing this?" Um if your workspace is

154:07

clean, the work that you do within that

154:09

workspace tends to be a lot cleaner as

154:10

well. And so, I mean, in my case I just

154:13

found what? Like 1 2 3 4 5 6 7 8 9 10 11

154:16

or something like that different things.

154:18

I've just sorted all these out now.

154:20

Anything here that is more personal than

154:22

business, let me know and I'll upload it

154:24

into the personal workspace instead. I

154:26

just let that go, but I don't obviously

154:28

want to show you cuz there are some

154:29

personal things in there. And that takes

154:30

me to the next point of workspace

154:32

organization, which is everything that I

154:34

just talked to you about when it comes

154:36

to like organizing with a business at

154:37

the top level and then having various

154:38

client folders in. You can do the exact

154:40

same thing with personal. And so, I

154:43

don't actually just have a business sort

154:45

of workspace set up. Claude has now gone

154:48

beyond just my business partner, okay?

154:50

And it also assists me with a lot of

154:51

personal stuff. When I say personal

154:53

stuff, I'm not referring to like, I

154:54

don't know, relationship troubles or

154:55

whatever. I'm talking about like, for

154:56

the most part, my health. Uh

154:59

you know, things like my my my

155:00

citizenship paperwork, uh you know,

155:02

important documentation relating to my

155:04

identity, personal projects that I have

155:06

that are, I don't know, related to like

155:07

learning piano,

155:09

uh that sort of thing. And so, like I

155:10

have like a business one over here,

155:12

okay? But just because I want this to be

155:15

really really clean, I'm also going to

155:16

show you guys a personal um version of

155:19

this.

155:21

Okay, which is basically the exact same

155:23

thing. And then instead of doing this

155:25

via clients, which you know, I mean like

155:28

obviously if it's it's a personal

155:29

project, it's not a client project

155:30

anymore and then you can't really do it

155:32

that way.

155:33

But instead of doing things based off of

155:34

clients, I now recommend doing things

155:36

based off of like domain and or

155:39

you know, like a particular field of

155:40

your life. So I haven't found the best

155:42

way to organize this yet, but for

155:44

instance, I have one right now on

155:46

citizenship because I'm currently

155:47

proving my my citizenship to you know, a

155:49

particular country in Europe and as a

155:51

result I'll be able to be a an EU

155:53

citizen, which is going to be pretty

155:54

fun. Likewise, I have a sub one called

155:56

health. This contains a couple of skills

155:58

that I use to like visualize my genetic

156:00

libraries and stuff like that. And

156:02

hopefully you guys are seeing the point.

156:03

What you do is you just sort of

156:05

you enumerate the clients of your

156:07

personal life, which tend to be projects

156:09

like citizenship, you know, your health,

156:11

I don't know, your skin care and

156:12

whatnot. And then you contact or or or

156:14

list those underneath your personal

156:16

workspace. Then you also have skills

156:18

related to your personal workspace like

156:19

hey, you know, can you clear out all of

156:21

my I don't know, like personal emails

156:23

for X, Y and Z. In this way, you have a

156:25

good separation, at least in my mind,

156:27

between business life, your personal

156:30

life and then also just logical grouping

156:31

of each of the different things that you

156:32

can do within them. So I also have as

156:34

mentioned, you know, that personal

156:35

folder. I can open that personal folder

156:37

at any time I want, which is right back

156:39

up here.

156:40

And that'll just contain, you know,

156:41

specific personal conversations I've had

156:43

with you know, Claude and anti-gravity

156:46

to do things. And I'm happy to like pay

156:48

token costs and stuff like that to

156:49

absorb that because my personal life

156:51

isn't like personal personal, it's just

156:53

stuff that is not business, right? If I

156:55

can improve the productivity of that,

156:56

might as well.

156:57

One more thing you'll notice is that

156:58

when I open up this personal, the colors

156:59

were a little bit different. Um I do

157:01

that on purpose. I do that because you

157:02

know, if I am working on business stuff,

157:04

I want it to be very clearly like

157:06

accessible and visible to like my my my

157:08

monkey brain. Like I instantly want to I

157:10

in my business folder. Whereas when I'm

157:11

in my personal folder, that's different.

157:13

And so what I've done is I've I've made

157:14

the outline of this green. I do that by

157:16

creating this .vscode settings folder,

157:19

and then I just have a sort of like this

157:21

config that VS Code reads at the

157:23

beginning of every run to like actually

157:24

change that upper bar. This isn't like a

157:25

super big unlock or anything, but I do

157:27

find just like having a slightly

157:28

different color will always just make me

157:29

want to be like, "Hey, this is my

157:30

personal folder, so I have access to

157:31

like personal information here, so I can

157:33

actually have a conversation about

157:34

whatever. I don't need to re-prompt it

157:35

with a bunch of stuff." You'll also

157:37

notice that you know, this doesn't have

157:38

like the Netlify or a bunch of those

157:40

other sections because this personal

157:42

folder only stores stuff that is like

157:44

for me. It's not for Netlify. Okay, so

157:46

hopefully that gave you some insight

157:47

into at least how I organize my

157:48

workspace, but this isn't by no means

157:50

the only way to do so. There are a bunch

157:53

of other ways to do it as well. One

157:55

candidate way is instead of having like

157:56

a business workspace, what you do is you

157:58

just enumerate all the projects in your

158:00

business. So I don't know, you might

158:01

have a project for instance that's like

158:03

website overhaul. What you do is you

158:05

have like a top-level folder. Okay, your

158:07

top-level folder might be business or it

158:08

might be whatever the name of your

158:10

company, Left Click Incorporated. Then

158:11

inside you have a projects folder, then

158:13

underneath your projects folder you have

158:14

like website design, you have

158:17

conversion rate optimization, you have

158:19

lead generation, and so on and so forth.

158:21

If you're running a business, you can

158:22

actually now have your CRM entirely

158:24

within Claude Code as like a .json file,

158:26

and then periodically on a daily basis

158:28

you can synchronize using some sort of

158:30

cron job or something like that to I

158:32

don't know, some events that are pulled

158:33

in from your calendar. You could store

158:34

stuff that way. I've seen people host

158:35

everything on GitHub as well, do some

158:37

sort of like daily uh

158:40

download or clone of GitHub, and then

158:42

some sort of like nightly push so that

158:43

they always have all their information

158:45

stored on the cloud. You can do that in

158:46

conjunction with the previous system I

158:47

told you about where the business /

158:49

personal / client one that I talked

158:51

about initially. You can also just ask

158:52

Claude to set it up according to however

158:54

you like. If you guys don't like the way

158:55

that I set up my workspace for whatever

158:57

reason, despite the fact that I do think

158:58

it is probably like top 10, um you know,

159:00

by all means you can just ask Claude,

159:02

"Hey, I want to have information for

159:03

this, I want information for this. Can

159:04

you build me like a a

159:05

naming scheme or a that will enable me

159:08

to do that better. Okay, hopefully you

159:10

guys like this and it made a lot of

159:11

sense to you. You guys have any

159:12

questions on that, let me know. Um but

159:14

let's move on to the next module.

159:16

Now on to a topic that I think a lot of

159:19

people don't like, security. And bear

159:22

with me. Usually most of the time when

159:24

people talk about security, it's sort of

159:26

divided into two camps. On the left-hand

159:28

side you have like the accelerationist

159:30

that are like, "Cloud code for

159:31

everything, baby. I just gave it my DNA

159:34

and USB stick with all of my personal

159:36

private information and passwords. Let's

159:38

do this thing." Then on the other side

159:40

you have like grubby old folk that used

159:43

to, you know, program computers by punch

159:45

cards. And so obviously there's some

159:46

irreconcilable differences there.

159:48

They're like, "What the heck? Why would

159:49

you even, I don't know, like make

159:51

something web accessible, man? You

159:52

should do everything on bare metal." And

159:54

then other folk are like, "Well, you

159:56

should just have cloud code do

159:57

everything."

159:58

Now the reality, like most things, is

159:59

nuanced and in my opinion the best case

160:02

is somewhere in between.

160:04

So this module and the next are going to

160:06

be a lot of talking and a little bit of

160:07

demoing. Um but it's important for you

160:09

guys to understand as cloud code ends up

160:11

becoming more of the predominant

160:13

generator of productivity in your life

160:15

that there are a few small security

160:16

differences or impacts that you can have

160:18

on cloud code that solve like 90-ish

160:21

percent of all of the possible downsides

160:23

and there's basically no reason not to

160:24

do them.

160:25

Okay, so I have this Google Doc over

160:27

here that I'm just going to walk you

160:28

guys through. And really the first point

160:30

I want to make is that everything on

160:32

planet Earth is hackable. It's always

160:33

just a question of how hackable.

160:36

You know, your front door is hackable.

160:38

Uh technically speaking the the

160:40

Department of Defense is hackable.

160:41

Everything is hackable. It's just what

160:43

is the risk and reward involved in

160:45

securing it to the point where you, you

160:47

know, dispel 90-ish percent of

160:49

attackers.

160:50

So

160:51

the way that I see things, you should

160:52

80/20 security,

160:54

avoid most of the low-hanging fruit, and

160:56

then just accept that there's always

160:57

going to be some small percentage of

160:58

people that are going to hack you anyway

161:00

or try to hack you anyway.

161:02

And you know, depending on how big your

161:03

vibe coded app or agentically engineered

161:06

flow ends up getting, obviously your

161:08

attack surface is going to increase

161:10

one-to-one with that.

161:11

You know, just for a reference, like

161:12

when I was first starting on YouTube, I

161:14

had like one login attempt per month,

161:17

and it was always me.

161:18

But now I get like probably 30 to 40

161:20

login attempts per day. It's just a

161:22

bunch of people that are constantly

161:23

trying to hack my ass. You know, back in

161:24

the day I had nothing sort of to lose,

161:26

wasn't a very big deal. Now it's

161:27

obviously a lot a lot bigger. And you

161:29

find this as you kind of go up the

161:30

chain. You know, if you become a public

161:32

figure or whatever, obviously you're

161:33

more likely to get that. Can't imagine

161:34

what Chris Hemsworth freaking open claw

161:37

probably looks like, but that's aside

161:39

from the point. Just know that

161:40

everything is sort of relative, and in

161:41

in your shoes, you should just cover the

161:42

80/20.

161:44

Okay? So, we're just going to get to a

161:46

point where our app or setup is less

161:48

hackable than the amount of time and

161:49

effort it would require to actually go

161:51

through it. Anybody could theoretically

161:52

break into your house right now. Most

161:54

people don't because there's just a

161:55

little bit more effort required to break

161:57

into your house versus, you know, if you

161:58

just unlocked your front door, then

162:00

somebody could walk right in.

162:01

So, what we're going to do is we're

162:02

going to put the equivalent of a fence

162:03

and a camera up, eliminate most of

162:05

these, and then we should be good to go.

162:07

Okay?

162:08

So, let's just cover some low-hanging

162:09

fruit right off the bat. And at the end,

162:11

I'm actually going to give you guys a

162:12

simple security audit that you guys

162:13

could use to copy and paste through any

162:15

sort of app or system or or website or

162:17

or web property that you have to

162:19

basically minimize the probability of

162:20

this occurring.

162:22

The first thing to know, which I think

162:23

most people don't, is that you actually

162:25

leak API keys every time you chat

162:28

through plain text with Claude.

162:30

Now, maybe they'll fix this at a future

162:31

version, but uh right now it's not. All

162:33

Claude code conversations are actually

162:35

stored in this folder right here in your

162:36

computer. Tilde just stands for home

162:38

folder {slash} and then dot is a hidden

162:40

convention in both Mac, Windows, and

162:42

Linux, where if you have a dot in front

162:44

of something, you know, you just can't

162:45

see it unless you specifically enable

162:46

like the hidden folder view.

162:48

So, what that means is you probably have

162:49

a a long-running log of API tokens that

162:51

are hard-coded there outside of, you

162:53

know, a dot env or whatever. And just to

162:55

show you, I'm going to head over to my

162:56

anti-gravity instance. this one is the

162:58

same auto research repo that we were

162:59

doing other stuff on, and I'm just going

163:01

to say, "Hey, I want you to remember the

163:05

word" Well, let's not even do that. I'm

163:07

just going to say, "Hey, what are your

163:09

opinions on Quetzal"

163:12

coatlus? I don't know. There's some sort

163:14

of animal I think called a

163:15

Quetzalcoatlus.

163:18

Um that's outside my wheelhouse. I'm a

163:19

coding assistant, so I don't really have

163:20

opinions on Mesoamerican feathered

163:22

serpents.

163:23

>> [laughter]

163:23

>> Interesting. So, hopefully I didn't

163:25

absolutely butcher this. Is it

163:26

Quetzalcoatlus?

163:28

Ah, okay. It's this right over here.

163:29

Okay.

163:30

So, I'm just going to insert this into a

163:32

chat history. And the reason why is

163:33

because I want to open this up, and then

163:36

I want to say,

163:37

"Search through

163:39

um dot Claude in the tilde folder for

163:41

any conversation mentioning

163:44

Quetzalcoatlus."

163:46

And what you'll see is there's actually

163:47

a long-running log of all conversations

163:50

basically right here in this folder. In

163:52

my case, it's /users/nextra. That's my

163:54

That's my home folder.

163:56

And now it's going to actually pull up

163:57

the conversation files and give it to me

163:58

word for word.

164:00

"Give them to me line by line, whole

164:02

convos."

164:04

And so essentially, you know, if we

164:06

actually uh dive into the output there,

164:09

um the way that this information is

164:10

stored is they're stored in JSONL files,

164:13

which are like JSON files that are line

164:15

by line by line.

164:16

And you can actually see how they're

164:18

returned

164:19

just by

164:21

doing a search here. I mean, I can

164:22

obviously open it up, but you know, I

164:23

probably have API tokens and stuff like

164:24

that. And then I don't really want to

164:25

do. You can see that they're organized

164:27

into um like a big JSON sort of

164:29

structure, right? And so you can

164:31

actually see if it pulls it out, you now

164:34

have the transcript which is user,

164:36

title, assistant, user, assistant. This

164:40

is the exact same chat that we just had

164:41

back here.

164:42

And so I'm sure you can imagine like

164:43

you're going to have a bunch of API keys

164:45

that you pasted in plain text also

164:46

available here.

164:48

And I mean like that's not the end of

164:49

the world. Obviously, we need to store

164:50

our API keys somewhere. But uh a very

164:52

low-hanging fruit in security is just

164:53

minimizing the number of places that you

164:55

have um the same sensitive information

164:57

spread out. Like, if you have the same

164:59

sensitive information, a key API key to

165:01

like your Anthropic account or whatever,

165:02

stored in five different places, the

165:04

probability somebody stumbles across

165:05

this at some point, if they're hacking

165:07

you or if it's just some sort of routine

165:09

data check or whatever, um is is like

165:11

not just five times higher, it's

165:13

something like 500 times higher.

165:15

And I think a lot of attackers now are

165:16

realizing the attack surface. And a good

165:18

place to like look for this sort of

165:19

thing is in the conversation history.

165:21

So, you know, you can't avoid having

165:23

some API key stored around, but a really

165:24

simple and easy way to avoid this is

165:26

basically instead of inserting um you

165:29

know, I'm just going to make like a

165:30

fake.env here,

165:32

and then instead, I think I'm going to

165:34

make a new conversation,

165:36

and instead of me just saying like,

165:37

"Hey, axolotl." Okay? What I'm going to

165:40

do instead is I'm going to store this um

165:42

animal_name,

165:44

and then we'll do axolotl

165:47

right over here.

165:48

I'm going to say,

165:49

"Hey, I just inserted an animal name in

165:52

an .env for a future task.

165:56

You know,

165:57

uh uh very important, we do not leak

165:59

this name."

166:00

Okay? Um now, what it's going to do is

166:03

it's just going to like clarify with me.

166:04

It can use this in some sort of function

166:06

or whatever the heck it wants.

166:08

And then, if I go through,

166:10

see how it says, "Never read or display

166:12

the contents of an .env file. Never

166:13

commit .env files to Git." That's

166:15

another pretty low-hanging fruit if you

166:16

um have

166:18

API keys stored in places that are not

166:20

your .env. A lot of people will

166:21

mistakenly push that to GitHub, and

166:23

like, you know, if you're pushing it to

166:24

GitHub, now it's on now it's on the

166:26

internet as well, right? Which is even

166:27

worse.

166:28

But, you know, now if I go over here,

166:29

and I say, "Hey, can you find me

166:31

conversations about um

166:34

axolotl in my and then I'm just going to

166:37

go .claud."

166:39

It's going to search all damn day long

166:40

looking for this thing, and it's not

166:41

going to be able to find it because we

166:42

haven't actually like specifically said

166:44

axolotl. And in fact, what's pretty

166:45

interesting is the only conversation it

166:47

found was where I specifically asked,

166:48

"Hey, can you find me an axolotl?"

166:50

So, it's going to look and see whether

166:52

or not it can find it in other

166:53

directories. It's not going to be able

166:54

to, but hopefully you guys get my point,

166:55

okay? Minimizing the attack surface in a

166:57

really simple way, just have all of your

166:59

API keys in a .env.

167:01

So, that's number one. Um number two,

167:03

low-hanging fruit, is that AI models

167:05

often hallucinate package names. In case

167:08

you guys didn't know, package names are

167:09

just like dependencies that you have to

167:11

pull in order for uh you know, the usage

167:13

of any project. Now, it is, you know,

167:15

like libraries and stuff like that. And

167:17

so, you know, there's like npm, which is

167:18

typically like the big package manager

167:20

here. And I'm just going to make this a

167:22

little bit more visible for you guys.

167:23

That stands for Node Package Manager.

167:25

But basically, like if you just type npm

167:26

install, okay?

167:29

Uh jeez, I don't even know. Like what

167:31

what are some popular libraries?

167:32

Anthropic? Maybe I'll just do a npm

167:34

search Anthropic. Okay, uh I don't know.

167:36

npm install @composio/anthropic.

167:40

Like basically, what occurs every time

167:41

you launch a new project or you have AI

167:43

like design something for you, is you'll

167:45

you'll go through online resources, this

167:47

big package manager, and then it'll

167:48

automatically install like all of the

167:50

packages it thinks it needs.

167:51

And like that's usually not that big of

167:52

a problem, right? Because uh npm is like

167:54

pretty well-vetted. But, you know, it's

167:56

a package manager, and so it manages

167:58

hundreds of thousands, millions of

167:59

different packages. And every now and

168:00

then one of these packages gets sort of

168:01

compromised. Now, the issue in the way

168:03

that uh this increases the attack

168:05

surface is that AI models often

168:07

hallucinate a package name. They won't

168:08

actually always get it right the first

168:09

time. Let's say uh you know, you want a

168:12

specific dependency or a package called

168:14

acorn, okay?

168:16

Sometimes, Claude, just because the way

168:18

that like the tokens uh were were sort

168:20

of baked into it, its various encoding

168:22

schemes and stuff like that, will

168:23

actually invent a dependency with like

168:25

an extra letter, acornS, like acorns or

168:28

acorn with an E or something.

168:30

And a lot of people that are sneaky and

168:33

terrible and super evil and malicious

168:34

have uh have sort of known about this

168:36

for a while because of like various

168:37

encoding issues and the statistical

168:39

probability of adding additional letters

168:40

and stuff. So, what they've done is

168:41

they've actually created new packages,

168:43

okay, with small little misspellings of

168:45

the main package. And they've made those

168:47

packages contain malware, things that

168:49

literally say, "Hey, I want you to go

168:51

through their .env and then go through

168:53

all of their

168:54

you know, tilde / .cloud conversation

168:56

logs and then send it over to me.

168:58

Wahaha."

168:59

And so, the idea there is, you know,

169:00

it'll obviously exfiltrate anything that

169:01

is important to you and then they'll

169:02

gain basically full control over your

169:03

account.

169:05

It's a form of like, I don't know,

169:06

prompt injection almost, but um

169:08

uh you know, if you're making any sort

169:09

of live projects or ones that tie to API

169:11

keys with any sort of unlimited usage,

169:13

you know, there are going to be some out

169:14

there where, I don't know, you just turn

169:15

the unlimited extra usage token thing on

169:18

and then you'll have access

169:20

theoretically to like billing tens of

169:21

thousands of dollars for a service.

169:23

Be very careful with that. You should

169:24

just audit your dependency list for any

169:25

unfamiliar package. You should actually

169:26

ask Claude like, "Hey, are there any

169:27

unfamiliar packages here that you don't

169:28

actually actively use all the time?" Or

169:31

you know, "Hey, before you instantiate

169:32

this the first time, I want you to take

169:33

a look at all at the npm run

169:35

and ensure that the only packages here

169:37

are like legitimate packages that have

169:38

verified histories and are not like

169:39

inserting malware. I'm kind of

169:40

concerned." And I'll give you guys like

169:42

a whole security audit you could use for

169:43

stuff like that in a moment. But the

169:44

point that I'm making is like this is

169:45

another attack vector, okay? A lot of

169:47

people don't realize this, but in

169:48

addition to leaking API keys and getting

169:50

it all over the place, AI models also

169:52

hallucinate package names.

169:54

The third main thing has to do with

169:55

databases and this is going to apply

169:56

mostly to people that are creating full

169:58

stack apps or apps that, you know, need

170:00

to call some sort of external data

170:03

store.

170:04

A lot of the time nowadays, to be

170:05

honest, I just store everything as JSON

170:06

files directly on my computer. It's a

170:08

lot easier and simpler for me because

170:10

I'm not really full stack end-to-end

170:12

apps as much these days. I'm for the

170:13

most part just designing flows for

170:14

myself or internal tools for my team.

170:17

But anyway, assuming that, you know, you

170:18

want to go a little bit further than

170:19

that, actually develop full stack

170:20

software apps.

170:21

Essentially, the simplest and easiest

170:23

way to ensure that like 90% of all noted

170:28

uh database breaches do not occur on

170:30

your app

170:31

is you just use this one little button

170:33

called row level security.

170:35

It's very straightforward. And basically

170:37

nobody does it, which sucks. So,

170:40

Superbase, which most of you are

170:41

probably going to be using for any sort

170:42

of vibe coded app function, does not

170:44

enable RLS by default. They probably do

170:46

so at some point, but for now, what that

170:48

means is if somebody signs up to your

170:50

app, um you know, typically they're

170:52

given a key by which they can access

170:53

their own database table.

170:55

Uh well, if they have a public key on a

170:58

database that does not have RLS enabled,

171:00

they can read, write, and delete every

171:01

other row in your database. And so, you

171:03

have a lot of cases where, you know,

171:05

there is some simple

171:07

uh I don't know. There was a a database

171:08

for like molt book, which was like

171:09

supposedly Facebook for AI agents. That

171:11

was just a few months ago. And, you

171:12

know, everybody was like, "My god, this

171:13

is the revolutionary whatever." And

171:15

then, like the most elementary of

171:16

security audits done by some

171:17

cybersecurity fellow showed that like

171:19

they did not have database or RLS, um a

171:22

real level security enabled on their

171:23

database.

171:24

So, he just went in and then he like

171:25

read literally every single AI agent

171:27

that had ever been created on the

171:28

platform in like 2 seconds. Then,

171:30

because he also had write access, he

171:31

created like a 100,000 fake AI agent

171:34

profiles in like 2 seconds.

171:35

Funny enough, uh Meta, Facebook actually

171:37

ended up buying them. And, uh hopefully

171:39

they understood that a big chunk of

171:40

those profiles were fake, but who knows?

171:42

Maybe they didn't.

171:43

The point that I'm trying to make is

171:44

like very, very low-hanging fruit. Takes

171:45

like 2 seconds to do. And, uh once

171:47

you're done with that, you can you can

171:48

kind of move on.

171:50

Okay, be wary anytime you're publicizing

171:52

a system like open claw, like your

171:54

little open claw uh package to the web.

171:57

So, let's say you have some open URL.

171:59

Let's say this is my open claw, okay?

172:00

And it's nickhappyfuntime.com. I'm kind

172:02

of curious, if I click on this, is there

172:03

anybody at nickhappyfuntime.com? Oh,

172:05

okay. Thank god, there's nobody at

172:06

nickhappyfuntime.com, because I'd

172:08

probably have to sanitize my eyes after

172:09

that.

172:10

Anyway, uh imagine you have your Claude

172:13

bot or molt bot or whatever the heck

172:14

it's called now on

172:15

nick-happy-fun-time.com.

172:18

Well, odds are, if you have a URL, and

172:20

it's like a short, straightforward URL,

172:22

and it's not an IP range that is like

172:23

owned by, I don't know, some virtual

172:25

private server hosting provider,

172:28

you are going to be queried constantly

172:29

by people that are looking for

172:30

vulnerabilities. They will be scanning,

172:32

okay? All over the place for every

172:34

single port that's currently open in

172:35

your computer. There are huge bot farms,

172:38

for instance, in China, in uh the

172:40

Philippines, in some Indonesian

172:41

countries, and obviously the West as

172:43

well. I'm not just trying to point a

172:44

finger over there. But, you know, that's

172:46

predominantly where a lot of these

172:47

attacks come from. And there's huge bot

172:49

farms that people have set up a long

172:50

time ago that literally that their whole

172:51

job is they just send tens of thousands

172:53

of requests per second to like every URL

172:55

constantly, scanning to see like, "Hey,

172:57

have they patched this one thing? Hey,

172:58

did have this security vulnerability?"

172:59

Hey, do they have this? And the second

173:01

even one of those things is good, like,

173:03

you know, allows them access, now they

173:04

have full access to your freaking

173:05

machine and box, basically, and then

173:07

they can do whatever the heck they want

173:07

with it.

173:09

So, I want you to know like if you set

173:10

up some sort of like public-facing

173:12

server using some sort of VPS-based

173:14

approach on uh you know, like Hostinger

173:16

or whatever the heck. Like one of these

173:17

like major hosting providers,

173:19

know that it is constantly going to be

173:21

tested.

173:22

And if you are like wild, you're raw

173:25

dogging this, you're wild westing this,

173:26

you don't like understand some pretty

173:28

foundational things about like firewalls

173:30

and um you know, RLS and and and so on

173:32

and so forth. Like, people will find

173:34

vulnerabilities, your stuff will be

173:35

hacked. And so, the idea is just make

173:37

sure to whatever you are putting in

173:39

there is not like super extraordinarily

173:41

sensitive. You know, don't give your

173:43

OpenClaw agent your social insurance

173:44

number or like a picture of your

173:45

passport or whatever. That to me is like

173:48

way too accelerationist, and I'm not

173:50

being the old grubby person yelling at

173:51

clouds in the sky being like, "Back in

173:53

my day we used to punch card stuff." I'm

173:55

just trying to be reasonable here,

173:56

right? Just no need to do stuff like

173:57

that for the most part.

173:59

You know, if you have like a a local

174:00

Claude instance that's running that's

174:02

authenticated through Telegram, and then

174:03

you're using like, I don't know, the the

174:05

the Claude channels feature or whatever,

174:06

probability that a hack will occur

174:08

[snorts] there is much, much lower

174:09

because you're just running it locally

174:10

and you're not actually connecting

174:11

through like an open thing, you're

174:12

connecting through a vetted uh uh you

174:15

know, Telegram kind of connector plugin.

174:18

But, if you're just like OpenClaw raw

174:19

dogging it, yeah, be be very careful

174:20

with that stuff.

174:22

By the way, this isn't just me ragging

174:23

on OpenClaw for the 4,000th time. I'm

174:24

trying to to reasonable about this. I

174:26

think decentralized autonomous agents

174:27

are obviously the future at some point.

174:29

But, you know, most of what we've seen

174:30

so far has literally just pissed away

174:32

people's API keys and credit card

174:33

information.

174:35

Speaking of credit card information,

174:36

never touch a credit card number. So, if

174:38

you guys are designing systems that

174:39

interface with any sort of credit card

174:40

whatsoever, don't actually like store

174:42

that data. Don't actually read that

174:43

data. If that data gets read at any

174:45

point by like an AI agent, hell, even

174:46

your AI agent, guess what's going to

174:48

happen? Well, same thing. You know,

174:49

you're going to leak those API keys.

174:50

You're going to stick them in your

174:52

conversation history. And then any sort

174:54

of hacker or you at any uh future point

174:57

in time if you misconfigure stuff, push

174:58

stuff to GitHub, or I don't know, like

174:59

uh uh uh

175:00

trade in your computer, whatever. You'll

175:02

now have like a big log of all of that

175:04

information just in plain text, which is

175:05

easily available. You know, a lot of

175:07

people will just like regex over your

175:08

entire computer looking for things like,

175:10

you know, credit cards if they get

175:11

access. And then what's a credit card?

175:12

Well, usually it's like

175:14

was it 16 or 20 characters or something

175:16

like that? I have to check my credit

175:17

card number, but it's like very very

175:19

stereotypical, right? You find 16 or 20

175:21

characters all connected together,

175:23

uh maybe like with a space in between.

175:25

Boom, you got yourself a freaking credit

175:26

card. Or maybe you don't even. They just

175:27

look for that length, then they check to

175:28

see whether or not it's like a Visa

175:30

pattern. If it is, you're screwed.

175:32

So, anyway, I guess what I'm trying to

175:33

say is like don't put that liability on

175:34

yourself by storing other people's

175:35

credit cards if you're running like some

175:36

sort of business thing. And then don't

175:38

um put that liability on your own card

175:40

by storing your own card um here.

175:42

You know, use services like Stripe. They

175:43

do everything for you. They are super

175:45

compliant, PCI compliant, and all this

175:47

stuff. They they they teams that just

175:48

like focus on making sure that stuff

175:50

that is stored on their servers uh never

175:52

get screwed up, and then you never

175:53

actually have to deal with like the

175:54

compliance regulatory aspect of touching

175:56

credit.

175:57

All right. Now, once you're done sort of

175:59

understanding this, which should be now

176:01

because hopefully nothing here was super

176:02

complicated. Although some of these

176:03

concepts are advanced, I understand. Um

176:05

all you need to do is just run anything

176:07

public-facing through some form of

176:08

security audit for like maybe the other

176:10

80/20. And so, this is a security

176:12

breakdown that I created for um

176:15

a Vibe coding course where I was showing

176:16

people how to make full stack apps. Uh

176:18

pretty cool using Gemini in case you

176:19

guys are interested. I guess it's is and

176:21

Cloud Code.

176:22

Uh you can find that on my channel if

176:23

you want to type like Nick's Drive Vibe

176:25

coding or something.

176:26

And uh

176:27

essentially down here at the bottom what

176:28

I have is I have a big security audit

176:30

prompt where you can actually just feed

176:32

this into Claude and then have it like

176:34

point out all the security issues with

176:35

whatever your your your flow is.

176:37

And so what I'm going to do is I'm going

176:38

to go back here to anti-gravity. And I

176:41

mean I sort of I don't really have like

176:42

anything that's public facing here, but

176:43

I'm still going to run it through auto

176:44

research.

176:45

Then I'm going to just create a new one

176:46

and I'll say

176:47

apply this to our auto research

176:52

flow,

176:53

um the one optimizing left click.

176:57

Once done the security audit, return me

177:00

everything we need to fix. I know

177:02

nothing is web accessible ATM.

177:05

Okay.

177:06

And so what this does is it's just some

177:07

it's just a big prompt that I developed

177:10

in conjunction with a bunch of AI

177:11

agents. I had to like read a bunch of

177:12

security blogs and so on and so forth

177:14

like look for the the biggest

177:15

low-hanging fruit and the simplest minor

177:16

configuration changes I can make.

177:18

And uh you know what it's going to do is

177:20

just go top to bottom and then apply

177:21

this.

177:23

The reason why I'm

177:24

spinning up a totally new conversation

177:25

history is because I do not want any

177:27

sort of conversation context to bias

177:31

what's going on here. I don't want the

177:32

same agent I used to develop my tool to

177:34

actually also run the audit because odds

177:36

are it's going to be biased and it's

177:37

going to do some specific it's going to

177:39

make specific errors cuz it's going to

177:41

think that it's better.

177:42

Do you see here how it's searching for

177:44

sk_live, sk_test, sk-bear and so on and

177:47

so forth? These are all API token

177:49

headers basically. These are like the

177:50

titles of API tokens.

177:52

What it just did there, other people are

177:54

going to do at any point in time if they

177:56

gain access to your system.

177:57

Same thing here with like model weights

177:59

and same thing here with like bash

178:00

scripts and stuff like that, okay?

178:03

Anyhow, so we're just going to read this

178:04

top to bottom. Um architecture summary

178:06

gives me some brief details about what's

178:08

going on. It's not a web app, it's a

178:09

local single GPU ML training pipeline,

178:10

that's easy. No hard-coded secrets, but

178:13

the git ignore does not include the

178:14

.env, .env.local and so on and so forth,

178:16

Okay? All the stuff that actually

178:19

applies here is going to be filled in.

178:20

So, in this case, this is an actual

178:22

failure, but in this case it's not as

178:24

not applicable cuz it's not an actual

178:25

web app.

178:26

Then you can see that there's also some

178:27

sections where it fails. So, finding

178:29

number one, supply chain low popularity

178:30

package, right? Supply chain issue.

178:33

Um let's see. Over here, it's failed on

178:36

some machine learning specific risks.

178:37

And it's sort of putting that out. It's

178:38

funny that it's using the term vibing.

178:41

I like that.

178:42

Anyway, so I'm not really going to go

178:43

through everything with you, but

178:44

basically what you do is you you you you

178:45

finish this. Then you just say, "Okay,

178:47

great. Fix according to your

178:50

suggestions."

178:52

Okay, and then once it's uh you know,

178:54

once it's done and whatever, I'm just

178:55

going to pretend it's it's done now even

178:56

though it obviously isn't. It's going to

178:57

take you like 3 or 4 minutes if you're

178:59

running on something that isn't like uh

179:01

you know, fast mode like I typically run

179:02

stuff on. Then what you do is you just

179:04

go through and then uh you actually

179:05

implement it. And just like I showed you

179:08

a moment ago to use something that is

179:09

not biased with the conversation

179:10

history, you spin up another agent to

179:12

take the recommendations then actually

179:13

go through and do it. Cuz you also don't

179:14

want that implementer agent to be biased

179:16

by the security audit kind of overly

179:18

constrained nature of it.

179:20

So, in that case you can use a sub agent

179:21

or some other model itself like Codex,

179:23

Gemini, or whatever.

179:24

And then, you know, ultimately you can

179:25

have it reviewed by Claude because I

179:26

think Claude is the best model. But in

179:28

this way you're basically like

179:29

diversifying similarly how we're

179:30

diversifying by putting seven out of 10

179:32

of our eggs in the Claude basket but

179:34

three out of the 10, you know, spread

179:35

across other models. You're diversifying

179:36

against any sort of inherent risk or

179:39

bias that Claude has

179:40

uh to work that is generated by other

179:42

Claude's versus, you know, Codex or

179:44

Gemini or whatever.

179:45

So, the best solution would actually

179:46

involve multiple runs through all of

179:47

them.

179:49

Okay, hopefully that makes sense. I

179:50

mean, I didn't want this to be a big

179:51

deal. Obviously, security as mentioned

179:52

is only a as big of a deal as you are

179:55

willing to make it because of

179:56

pre-existing assets and what you have to

179:57

risk and stuff like that.

179:59

So, if you just understood what I talked

180:00

to you about right here, and then if you

180:02

get, you know, a security prompt like

180:04

what I showed you here, um you you

180:05

should be good. Just pass something like

180:06

that through an AI agent after you're

180:08

done a project and it'll like cover most

180:09

of the hanging fruit.

180:10

And by the way, if you want that

180:11

security audit, then definitely check

180:12

out that um Vibe Coding full course.

180:15

Really easy, just type Nick Rive Vibe

180:16

Coding. I actually give you guys all

180:17

that information for free there. You can

180:19

also watch it if you want to learn how

180:20

to develop things with other models.

180:23

Congratulations, you made it to

180:24

essentially the end of the informational

180:26

Claude technical content of the course.

180:29

And now I just wanted to reserve maybe

180:30

10 or 15 minutes to chat a little bit

180:32

about what I consider to be the future

180:34

of Claude. Uh not just the future of

180:36

Claude code, but the future of Claude

180:37

the model, as well as the future of just

180:39

agentic engineering in general.

180:41

And the reason why I talk about this is

180:42

because it's a topic that's very close

180:43

to my heart. I've been considering this

180:45

for probably the last 10 or so years. As

180:47

a kid they grew up on science fiction,

180:49

you know, um

180:51

Foundation from Asimov, tons of Arthur

180:53

C. Clarke books and Heinlein and so on

180:56

and so forth. I I've thought a lot about

180:58

like what the far future would look like

181:00

in an environment that is controlled by

181:02

agents like Claude code. And I've also

181:04

thought about some of the intervening

181:05

steps we need to get there. And now that

181:06

it's sort of being thrust in my face, I

181:08

think there's a lot that you could

181:08

realistically learn from you just like

181:10

fictional representations of this. That

181:12

uh most people who probably haven't just

181:13

stuck their head so far in the science

181:15

fiction bubble, um I think uh you know,

181:17

I think would find value in hearing.

181:19

In addition, I obviously have a lot of

181:20

exposure to both mid-market and then

181:22

enterprise here, not to mention all the

181:24

small businesses that I work with um

181:26

through Left Click. And I think that

181:27

gives me sort of an edge here to at

181:29

least give you guys a some sort of

181:31

plausible future that has more of a 10%

181:33

chance of probably being true. I mean

181:34

like things are changing so quickly, I

181:35

obviously can't be 100% sure what is

181:37

going to occur. But these are some

181:38

things that I consider to be like pretty

181:40

low-risk bets that if you make, you'll

181:41

probably have some form of alpha.

181:44

Okay, so the first main one is this

181:47

trend of decreasing human involvement.

181:50

Do you guys remember earlier when I

181:51

showed you guys that diagram where it

181:53

was like Vibe Coding to agentic

181:55

engineering to basically like

181:57

research-based direction with auto

181:58

research and and frameworks like that

181:59

coming up?

182:00

Well, this is still something like we

182:02

are creating, right? It's sort of like

182:03

open-sourced, not not open-sourced, but

182:06

um it's something that like, you know,

182:07

the community is sort of working on. But

182:09

all of these approaches are soon to be

182:10

quite formalized. And it is very likely,

182:13

in my opinion, that we are going to

182:15

continue decreasing human involvement in

182:17

tasks. This auto research thing is a

182:19

great example of ways to, you know,

182:20

democratize sort of like little

182:22

improvements. I've kept this auto

182:24

researcher running, by the way. If you

182:26

guys have remembered from like a couple

182:27

of launches ago, and we're now actually

182:29

at like almost 8,000 millisecond load

182:31

time from a baseline of 1802.

182:33

Imagine if you had this running from

182:35

3,000 days in a row or whatever. Or if

182:37

you had this running at like inference

182:39

capacities 100x this, right? Which we

182:41

obviously getting to.

182:42

You guys remember how slow GPT-3 was

182:44

back in the day if anybody here is an

182:45

old head that used that? Well, GPT-5.4

182:49

faster instant or whatever is way

182:50

faster. And imagine if you had a model

182:52

that's 100 times that that fast with the

182:53

same level of intelligence. You can make

182:54

some major updates to basically

182:56

anything.

182:57

And so the idea is, you know, we're

182:59

probably not going to increase the level

183:01

of human involvement in like direct

183:03

coding and stuff like that. Which is

183:04

fine. I'm not making like a value

183:05

judgment or a normative judgment here.

183:07

Um, but I imagine you as a developer or

183:10

business person or whatever will

183:11

actually probably grow less involved in

183:13

the day-to-day work of either your

183:16

company, your research lab, your your

183:18

your your your app, whatever the heck.

183:20

And so my take is, in the future, we're

183:23

going to move towards this sort of thing

183:24

that a lot of frameworks have tried to

183:26

formalize, which is that we're each

183:27

going to be the CEO of sort of like our

183:29

own company. Whether it's an actual

183:32

company in practice or whether it's, you

183:33

know, some sort of organization that's

183:35

like a company. All of us will basically

183:36

be just like the the chief executive

183:38

officer running teams or fleets of

183:40

agents that are constantly doing things

183:41

on our behalf and that have some sort of

183:43

formalized framework that also like

183:44

helps them optimize and and make better.

183:47

And so

183:48

sort of the the way that this works, I

183:50

imagine, is we would go from, you know,

183:52

like the old school Wright brothers

183:54

flying the plane ourselves to sort of

183:56

like modern aircraft engineers, where

183:58

there's somebody in the cockpit, but for

184:00

the most part, you know, an autopilot is

184:01

taking over the vast majority of the

184:03

work. Even in, you know, like takeoffs

184:05

and landings now, they're obviously like

184:06

so much so many SOPs and so much like a

184:08

a process and framework that, you know,

184:10

you can imagine how a system that was

184:12

much less developed than ours, much less

184:14

capable of deep thinking and stuff,

184:15

could actually probably just execute it

184:16

entirely at this point. That said, you

184:18

know, will we ever get rid of a human in

184:20

the loop to some capacity? There are

184:22

just so many regulatory blocks and I

184:24

think like ethical issues with that that

184:26

we will probably always just have some

184:27

person like manning a ship. It's just

184:29

the number of ships that a person will

184:31

man, the number of of discrete agents

184:33

will just continue increasing. And so,

184:35

you know, rather than have a hundred

184:36

people do a task in some specific

184:38

company like we used to have, we might

184:39

have one person do a hundred tasks.

184:41

Leverage will go up.

184:43

Now, a good example of this is Claude's

184:45

recent

184:46

auto mode. I don't know if you guys have

184:47

seen I said auto mode, but I don't know

184:50

if you guys have seen their recent

184:51

development. Where basically you now

184:52

have the ability to run some sort of

184:54

autonomous mode instead of choosing, you

184:57

know, switch permissions or sorry, uh uh

185:00

execute uh um

185:02

uh bypass permissions down here or ask

185:04

it before edits or edit automatically

185:06

and and so on and so forth. Well, now we

185:08

basically have an additional one, auto

185:09

mode, which um I just can't see here

185:11

right now because I'm using a slightly

185:13

older version of Claude. So, I don't

185:14

have that yet. But basically, you know,

185:15

instead of you actually having to like

185:16

go through this whole process of

185:17

changing the security

185:19

um changing the access that it has, you

185:21

know, Claude just does that for you. So,

185:22

like that's a pretty good example of

185:23

something that used to require a person

185:24

and now it's just like, well, Claude's

185:25

going to get it ninety-nine point nine

185:26

percent of the time, screw it. I'll I'll

185:27

give it to them.

185:29

Okay, so that's a very small microcosm,

185:31

but like imagine the rest of the loop,

185:32

like the planning loop right now.

185:34

Typically, you have Claude develop a

185:35

plan for you and then you implement on

185:36

that plan. Um that whole thing is just

185:38

like being internalized. Like we're not

185:39

actually doing most of the plan

185:40

development now. And we we will not

185:42

continue to do most of the plan

185:43

development in the future.

185:44

Realistically, Claude's going to do both

185:45

the planning and the implementation.

185:46

Then the Q&A, it's like right now we're

185:48

sort of in the loop. We're sort of like

185:49

clicking in the buttons, running it.

185:51

Well, they're developing automated

185:52

testing procedures where Claude actually

185:53

also does the Q&A for you and then

185:55

delivers you the whole thing.

185:56

And so, some people hate this because

185:58

they're like, "Well,

185:59

they're taking my jobs and whatnot." And

186:01

I think there's I think there's a fair

186:02

point to that, you know, human beings

186:04

leverage will continue to increase, but

186:06

depends on like how much work is there

186:07

really to do, how many software products

186:08

are there really to develop, do we

186:09

actually are we even going to have like

186:11

the demand for that sort of thing? I

186:12

think that's like a reasonable

186:13

conversation to have.

186:15

I don't, you know, unfortunately I don't

186:16

know the answer. My my take is like

186:17

eventually we're probably going to have

186:18

to move to some sort of different

186:19

economic system because most of the

186:21

world would be unemployed otherwise.

186:23

But that's me getting all political.

186:25

Um that's number one, okay? So, the

186:26

trend of decreasing human involvement is

186:27

very likely to continue with cloud code.

186:29

They're now at the point where they're

186:30

developing this so rapidly that like AI

186:32

is helping AI design products.

186:34

And you know, automotive is just the

186:35

beginning of like I think a massive

186:37

suite of rollouts that will

186:38

significantly improve your experience.

186:40

But, you know, make you more hands-off.

186:42

My second one is more of like an

186:43

economic

186:45

consideration, which is that software

186:46

products and tools, okay, the the

186:49

quality of the things that you build

186:51

will no longer be the moat.

186:53

So, in the in the past, in the good old

186:54

days, back when I was on the come up,

186:56

how good your software was,

186:58

think like Windows, think like, you

187:01

know, like macOS,

187:02

how good that operating system was,

187:04

might have been the only thing that

187:06

distinguished that operating system from

187:08

another operating system. And if it was

187:10

really, really good, then obviously it

187:11

would be much more popular and then it

187:12

would get, you know, a bunch of like

187:13

inherent interest and stuff like that

187:15

cuz the capabilities and you'd obviously

187:16

use it.

187:17

So,

187:18

the issue with that nowadays is you can

187:20

make Netflix in 5 minutes.

187:23

Netflix before was this innovative

187:24

streaming model that, you know, was like

187:26

wow, you know, you could just load the

187:27

thing and then the the the the video

187:29

loads on you for for you on demand and

187:32

it's incredible and like the streaming

187:33

and latency and uptime and all that

187:34

stuff. It's like super proprietary

187:35

technology.

187:37

Well, now it's like I can code Netflix

187:38

in 5 minutes with like, you know, three

187:39

or four agents on Fast One. So, it's

187:41

like what is the value of Netflix?

187:43

What is the moat that differentiates

187:45

Netflix as sort of like this like old

187:47

school medieval castle from all of the

187:48

attackers that, you know, could actually

187:50

take it down?

187:51

Well, the moat now, and uh this has been

187:53

something for at least a couple of

187:54

years, the moat now is no longer the

187:56

software. It is the distribution. So, in

187:58

a world where everybody has basically

187:59

like a

188:01

I don't know, a a nuclear weapon, is the

188:03

differentiator like everybody has a

188:04

nuclear weapon? No, the differentiator

188:06

moves to other things like uh I don't

188:08

know, the political framework, like the

188:09

wellness of the populace, and stuff like

188:10

that. What I'm trying to say is like

188:12

that that skill, that software

188:14

engineering ability is no longer going

188:15

to be the moat. And instead, um the moat

188:17

is going to move to, you know, the

188:19

connection step that a company has to

188:21

its consumers, the reputation that the

188:23

company has in the market, the

188:26

distribution that it has with a bunch of

188:27

vendors that, you know, are hard-won

188:29

relationships and connections that they

188:30

realistically built over the course of

188:31

many years. You know, Netflix now has a

188:33

bunch of patents and and rights and uh

188:35

licenses and stuff like that to air

188:37

specific shows. It's seen this coming

188:39

and and so it's tried to diversify

188:40

accordingly. But you're going to see

188:41

that in basically every software

188:43

platform. The moat will like probably

188:45

move more to the distribution and the

188:46

legal and compliance aspects than

188:47

necessarily like how good the software

188:48

is. Which means you're going to have

188:50

like these cracked, probably like 14,

188:52

15-year-old kids designing like the most

188:53

incredible, amazing software ever, and

188:56

then that software will be able to

188:57

reproduce anything that like a major

188:58

business would do in like a hundredth of

189:00

the time.

189:01

But, you know, because they don't have

189:02

like the compliance or or whatever

189:03

certifications or whatever, you know,

189:05

it'll probably be more difficult for

189:06

them to actually go to market with

189:07

something like that, despite it being

189:08

like objectively superior.

189:11

And uh you know, the way that I see it

189:12

is we already have AI models that are at

189:13

the limit of human reasoning capability.

189:15

They can run hundreds of times faster

189:16

than our brains, soon to be thousands of

189:18

times faster than our brains on basic

189:19

tasks. So, even if they're not like

189:21

better than us at the software design

189:23

individually, if you run a thousand, you

189:25

know, 90 IQ models comparatively like

189:29

one to 100 IQ human, those will

189:30

eventually figure out the things that

189:32

that one 100 IQ human would do. And not

189:34

only will you develop more software like

189:35

quality, you'll also develop more

189:36

software quantity. And so, software as a

189:38

just a market thing, supply and demand,

189:40

like economically, the supply will be so

189:41

damn high that the demand for any sort

189:43

of like purchasable software gets a lot

189:45

lower.

189:46

Which means I personally don't think

189:47

like a SaaS product is really the play

189:48

here. I don't think there's going to be

189:50

any sort of life cycle for like

189:51

subscription-based products. I think

189:52

you'll have a short window of time where

189:54

you could actually just monetize like a

189:55

one-time buy product, and then most

189:57

people will just say, "Well, should I

189:58

spend $199 on the product or should I

190:00

just spend $19 plus 30 minutes of my

190:02

time on tokens and they just design it

190:04

for myself?"

190:05

And I think that's going to change the

190:06

way that we do, you know, like software

190:08

more generally.

190:09

So, I'm not very bullish on like, you

190:11

know, developing software as a service

190:12

apps and stuff like that. I have a lot

190:13

of people be like, "Nick, you know all

190:15

this stuff, you know how to design all

190:16

the software. Like, why aren't you

190:17

making a software app and why why aren't

190:18

you monetizing your community, let's

190:19

say, through software?" And I'm like, "I

190:21

don't really be able to do that for a

190:22

short period of time. And then even if I

190:24

were to, like, where's the value in that

190:25

if anybody could just make it? I'm

190:26

sending them like 20 minutes and a

190:28

couple bucks in tokens, right? It's not

190:29

that big of a deal."

190:31

So, I mean, I would I would move

190:32

accordingly, I guess.

190:34

Okay, so the third thing that I'm like

190:35

99.9% sure of is that the pace of change

190:39

is not slowing down anytime soon.

190:41

It will continue to accelerate. Just as

190:44

technology has helped us

190:46

increase the pace of change throughout

190:47

history with things like the printing

190:49

press, uh with uh developments and, you

190:51

know, communication with like the

190:52

telegraph and so on and so forth.

190:55

You know, these things don't just

190:57

improve the quality of life of the

190:58

average person, they improve the

191:00

research and development arm of

191:02

technologists who work on that exact

191:04

thing. And so, because of that, you

191:06

know, the pace of change is is basically

191:08

just going up.

191:09

If I had to um graph sort of where we

191:11

are now,

191:13

and I will,

191:15

because I freaking love graphs, right?

191:17

>> [snorts]

191:18

>> Just the best.

191:20

And if I were to graph the intelligence,

191:23

which is a very loose term here and

191:24

obviously it means different things to

191:25

different people, but the intelligence

191:27

of a model over time,

191:29

you know, basically, I'd go like this.

191:32

Okay? And so, this back here was sort of

191:34

like linear growth from like maybe like

191:36

the 1970s

191:38

and stuff with like Minsky uh you know,

191:40

1970s and 80s and stuff. Minsky and like

191:42

the the first few neural nets and stuff

191:43

like that. Then this right over here is

191:45

probably like I don't know, 2010 when

191:48

models started actually doing stuff,

191:49

right? Then this over here

191:51

it's like 2020.

191:54

Yeah, this over here

191:57

is like

191:58

2020

192:00

five and then

192:01

this over here is 2026. Do you see how

192:04

how like high

192:05

this is going, how quickly? And then um

192:08

a point that I want to make is basically

192:09

like

192:10

this right here is the intelligence of

192:12

maybe like a like a chimpanzee.

192:15

Okay? Um this right here is the

192:17

intelligence of like an average human.

192:19

And then this right here

192:21

is maybe the intelligence of like

192:22

Einstein.

192:24

>> [gasps]

192:24

>> And um what we what we have now is, you

192:27

know, we're we're like right over here,

192:28

man. These models I I say smart as a

192:30

chimpanzee not to didn't diminute or

192:33

whatever chimpanzees, but um you know,

192:35

their brains are extraordinarily

192:36

advanced and developed. They have like

192:37

these cerebelli, these these sections of

192:39

their brains that are responsible for

192:40

calculating like millions of of

192:42

movements and and so on and so forth

192:44

every minute.

192:45

Like it's a very complicated thing to

192:46

like replicate the intelligence and

192:47

distributed intelligence of an organism

192:49

and you don't capture that all just by

192:50

like, "Hey, can it write? Hey, can it,

192:52

you know, reason and do math?" Have you

192:53

ever seen like a chimpanzee's like

192:55

memory? Have you seen it's like ability

192:57

to like uh you know, move around on a

192:59

page and like uh figure out symbolism

193:01

and then uh symbols, sorry, and then

193:02

like count numbers up in their motor

193:04

neurons. Anyway, the point I'm making is

193:06

not this is a course on chimpanzees, so

193:08

I'll stop talking. God, that's my nerdy

193:09

side.

193:10

But um that the gap between the

193:12

intelligence of a chimpanzee, if you

193:13

just count up all the neurons in its

193:14

brain, the intelligence of a human, if

193:16

you count up all the neurons in its

193:17

brain, the intelligence of Einstein,

193:18

they're actually very close together.

193:20

They're very clustered. And I'd say like

193:21

we're basically right over here right

193:22

now. So guess what's going to happen in

193:23

like, you know, the next few years? This

193:26

is going to go like up here. And we are

193:27

going to it's going to be like, "Wow,

193:29

these things are so dumb. They're dumb.

193:30

Oh, wow, cute. They can do things that a

193:31

chimpanzee can do." And then like six

193:33

months it's like, "Oh, okay, these

193:34

things are now like

193:35

you know, freaking galaxy brain

193:37

intelligences that you know, can do

193:38

everything and anything for us.

193:40

And um imagine what happens when, you

193:41

know, all of this is just humans working

193:43

on stuff.

193:44

And then eventually it gets the point

193:45

where it can actually like use human

193:47

level intelligence, which is right now,

193:48

to like improve its rate of growth. This

193:50

thing is just vertical. I mean, this

193:52

thing would go so vertical it'd go

193:53

through my roof in 2 seconds.

193:55

So, that's my take on it personally. I

193:57

think um

193:58

you know, I think we're getting really,

193:59

really close to super fast paces of

194:01

change. And if you guys have like been

194:02

monitoring the the Claude even Claude

194:04

code X page recently or like seeing

194:06

YouTube, there's there's new updates

194:07

coming out every day. This would have

194:08

been unfathomable just like 3 or 4 years

194:10

ago to make this level of development

194:12

and this level of like small additions

194:14

to a software product while also making

194:15

sure they're testable and reliable.

194:17

It's just because intelligence is making

194:19

intelligence more intelligent now.

194:22

And then the last thing I'm going to say

194:23

is that uh the people that

194:26

will control, not necessarily control,

194:28

but have the most like power and ability

194:30

over the course of the next years are

194:31

people that learn to use this technology

194:33

now. You're part of a very like

194:34

privileged minority and I don't say that

194:36

in like the political sense of the term

194:37

cuz I think it's all muddled up. But

194:39

like you're part of a minority of people

194:41

right now that like actually use this

194:42

technology. Do you know how few people

194:43

even understand what an agent harness

194:45

is? We're talking like sub like 1% of

194:47

the population of Earth.

194:49

The percentage of people that know how

194:50

to use a an agent harness, like you are

194:52

doing right now, is even less. It's a

194:54

fraction, vanishingly small percentage.

194:56

I don't know if everybody that watches

194:58

this is old enough to remember, but

194:59

there were like some protests back in

195:00

the day on Wall Street. And the point is

195:03

that they were like, "We are the 99% or

195:05

whatever." And they were protesting the

195:07

massive wealth divide in specific parts

195:09

of America between like, you know,

195:11

really, really wealthy people that work

195:12

on Wall Street and then like the

195:13

populace, the rest of the people that

195:14

like I don't know, manage the service

195:15

industry and hospitality and basically

195:17

do everything else. And they're like,

195:18

"Why do you guys get to have like

195:19

thousands of times more money than us?"

195:21

Um you are the 1% right now.

195:24

You are that group of people that I'm

195:26

sure in the future other people will be

195:27

raising their hands about and you know,

195:29

shaking their fists at. Because you have

195:32

an enormous capability to use models

195:34

like this for just cents on the dollar

195:36

to do incredibly amazing economically

195:38

valuable things that would take that

195:40

other group of 99% like like months to

195:42

do what you could realistically do in a

195:44

day.

195:44

It's insane. You know, I think you can

195:46

talk all day about like the wealth

195:47

divide, but you can also talk about like

195:48

the productivity divide. And the wealth

195:51

improves the likelihood that you will be

195:52

in that product the positive chunk of

195:54

the productivity divide.

195:56

You right now, even if you don't have a

195:58

lot of money, have access to insane

195:59

technology and leverage simply because

196:01

you're in a

196:02

So, that's going to increase.

196:03

Now, William Gibson, one of my favorite

196:05

authors, said it best, "The future is

196:06

here, it's just unevenly distributed."

196:08

Meaning that like we have access to

196:09

insane technology, it's just like not

196:10

all of us do at the same rate.

196:13

There's small pockets of people like

196:14

yourself that understand how to use

196:15

these tools far better than others, and

196:16

in doing so, you have the ability to

196:17

reap asymmetrical rewards over a small

196:19

chunk of time.

196:20

And my take is as the economy shifts to

196:21

accommodate smarter than human

196:23

intelligences,

196:24

the people that understand things like

196:26

agent harnesses and coding harnesses,

196:27

people that understand how to use the

196:28

best models in the world like Claude,

196:30

uh you know, Opus or Mythos or whatever

196:32

the heck we're at now.

196:33

People that know how to turn these into

196:34

economically valuable things are the

196:36

ultimate people that are going to win

196:37

this share of the future, um whatever

196:40

small percentage it is. Because given

196:42

the massive unbounded upside here, like

196:43

we're talking

196:45

you know, solar panels orbiting the

196:47

freaking sun in a few years like we're

196:49

we have solar panels, but the point that

196:51

I'm making is the massive potential

196:53

upside of if everything goes right with

196:55

this technology, things don't go super

196:56

wrong.

196:58

If you own even 0.00000000001%

197:02

of that potential future because of some

197:04

decisions that you made today to, you

197:05

know, upskill and start this

197:07

productivity kickoff,

197:08

um you know, like the the the abundance

197:11

of your own personal life would would be

197:12

huge.

197:13

Okay, so

197:15

I guess that's it. We made it to the end

197:16

of the course, and that's really all I

197:17

have to say on that.

197:18

Hopefully you guys appreciated learning

197:20

everything that I had to give on Claude

197:22

code, and you guys have learned some

197:23

advanced concepts here, whether it's

197:25

about, you know, initial system prompts

197:28

and and and Claude done on these or it's

197:30

some of the more obscure things and

197:31

esoteric things like security or the

197:33

future like I just talked about.

197:35

Um, if you guys like this sort of thing,

197:36

you'd be doing me a big solid to

197:37

subscribe to the channel. For whatever

197:39

reason, something like 70% of my regular

197:40

viewers are not subscribed. I think it's

197:42

just how YouTube works. Most people

197:43

don't sub. But, you could you could sub.

197:45

That would really help me out. I want to

197:46

get this sort of message out to more

197:47

people and obviously help them be in

197:49

that small little chunk.

197:51

Uh, if you do me a solid, leave a

197:52

comment down below with video idea or

197:55

something that you want me to cover. I

197:56

actually get most of my ideas directly

197:57

from my audience now, so I'd really

197:58

appreciate that.

198:00

If there's anything that I didn't cover

198:01

here or maybe didn't touch on that you

198:02

would like me to touch on or maybe

198:03

anything that I personally made a

198:04

mistake on, I'd I'd love to hear it

198:06

because I'm trying to improve my ability

198:07

to use these tools. Finally, I also help

198:09

other companies implement this sort of

198:11

thing in their own businesses, whether

198:12

you are a small to mid-size business,

198:13

mid-market, or enterprise. Um, so if you

198:15

want to chat with my team, just check

198:17

down below. Uh, somewhere at the top of

198:18

the description, there'll be a link.

198:19

Thank you for making it all the way to

198:20

the end of the video. I'll see y'all

198:22

soon. Bye.

Interactive Summary

This course provides an advanced guide to using Claude Code and AI agents to optimize business productivity and development workflows. It covers essential topics such as optimizing Claude.md system prompts for knowledge compression and capability declaration, advanced workspace organization, agent harnesses, and parallelization techniques like agent teams and sub-agents. Additionally, it addresses security best practices, auto-research methods for iterative performance improvement, and provides practical advice on automating tasks through HTTP requests and browser automation.

Suggested questions

5 ready-made prompts