HomeVideos

I Fixed Claude Without Touching Any Code

Now Playing

I Fixed Claude Without Touching Any Code

Transcript

1582 segments

0:00

You might have noticed that I haven't

0:01

been posting videos as much recently.

0:02

Part of that's cuz I had a lot of fun

0:04

going to open sauce and that definitely

0:05

threw off my schedule for recording, but

0:07

the bigger part is just that I've been

0:08

coding way, way more. Way more. I've

0:12

landed dozens of PRs in just the last 3

0:14

days and many more if you're willing to

0:16

go back a week or two. I have been

0:18

coding more than I ever have and more

0:20

importantly, I've been landing more code

0:22

than I ever have. A lot of my

0:23

productivity wins recently can be

0:25

credited to T3 code. We've made a ton of

0:27

improvements that make it way easier to

0:29

work on lots of things for real

0:30

applications in parallel, but I've also

0:33

spent a lot more time in my editor. The

0:35

last 2 days I've spent more time in VS

0:37

code than I probably have in the last

0:38

year actually writing and reading code.

0:41

As long as you consider markdown files

0:43

code. I know this sounds insane, but I

0:45

actually sat down and spent maybe 6

0:48

hours just editing markdown and then the

0:52

next day spent another 6 to 10 hours

0:54

testing those changes and making small

0:56

adjustments afterwards.

0:58

And I want to share how much this has

1:00

benefited me. I am genuinely annoyed at

1:03

how much these small changes have

1:05

improved my development experience and I

1:07

have a feeling they will help you out,

1:09

too. I want to cover everything that

1:10

I've been doing here from how I actually

1:12

make these changes and test them to how

1:14

I keep these things synced across the

1:16

many different machines I do dev work on

1:18

cuz there's like five of them now to how

1:20

I actually write these things and audit

1:22

the histories I have with my agents to

1:24

make sure I'm doing them right, but I

1:26

got one more really helpful thing I want

1:27

to show you first. Today's sponsor.

1:29

There's a lot of parts to your app that

1:30

used to be really easy to build before

1:32

AI that got even easier thanks to agents

1:34

going and doing all the work for you.

1:35

And if I'm being honest, off is

1:37

definitely one of those things. So,

1:39

what's the role of an off platform like

1:40

WorkOS if agents can just build off for

1:43

you? There must be a reason why

1:44

companies like OpenAI, Anthropic,

1:46

Cursor, Perplexity, Vercel, Replit, Bolt

1:48

and everyone else in this list are

1:49

betting on what WorkOS built, right?

1:52

Well, yeah, of course, but it's probably

1:54

not what you think. It's not because

1:55

setting up off is so hard, it's because

1:57

your auth needs to do more than it ever

1:59

did before. Auth is how you know what

2:01

users are accessing your things. Auth is

2:03

how you set up an enterprise to use your

2:05

service. Auth is how agents can do

2:07

things on behalf of their users. You

2:09

could try to set up all of these pieces

2:11

yourself, but if you're planning on

2:13

DIYing it and you want to sell to real

2:14

businesses, you should probably go look

2:16

at the admin portal on WorkOS, because

2:18

you're going to have to go set up Okta,

2:19

SAML, Duo, ADP, ADFS, and all of these

2:23

other things that I know you don't want

2:25

to set up. You're watching this channel,

2:26

you like writing real code. Don't deal

2:28

with this, just use WorkOS. And when it

2:30

comes to agents, WorkOS is helping

2:32

pioneer a new standard to allow for your

2:34

agents to auth on a service, which is

2:36

huge. They're partnering with a ton of

2:38

companies like Cloudflare, Firecrawl,

2:39

Resend, Monday, Kernel, and more in

2:41

order to get this standard ready, and it

2:44

already is. So, if you have a small app

2:46

with a handful of users, no businesses,

2:48

and no agents using it, you're fine

2:49

rolling your own. But, if you want to

2:51

build something serious, look no further

2:53

than soydev.link/workos.

2:55

So, as I mentioned before, I've been

2:56

coding a lot more, and I actually

2:58

decided to sit down and take the time to

3:00

streamline certain things that I was

3:01

doing. Since T3 Code makes it so easy to

3:04

work across multiple machines, I have

3:07

four that I work on T3 Code itself on in

3:09

T3 Code. You can literally just click

3:10

and choose what box is going to run that

3:12

diff. It's super convenient to have lots

3:14

of machines that you can run different

3:15

work on depending on the needs of the

3:17

work you're doing. Managing everything

3:19

that exists on all of these machines and

3:20

making sure skills are available on the

3:22

right ones and not available on

3:23

different ones not trivial. And I caught

3:26

myself starting to make a custom system

3:29

to manage all of that. And I got pretty

3:31

far into it, and it was buggy and

3:33

annoying and had lots of annoying edge

3:35

cases. So, I stopped. This directory

3:38

we're looking at here is my fleet repo.

3:41

In my fleet repo is just a bunch of

3:43

markdown files and a fork of vibe proxy,

3:45

cuz I'm making a lot of changes to the

3:47

CLI proxy API for my specific needs. And

3:50

in my agents in Cloud MD, I describe

3:53

what this project is and what I want

3:55

agents to do in it. So when I do make

3:58

changes to the skills I have, like I

3:59

have my core agents in Claude MD and I

4:01

have the skills directory that has a

4:03

folder for Claude only skills, a folder

4:05

for my command center skills, which is

4:06

just this one machine, the leader

4:08

machine that controls everything else,

4:10

and a universal folder for the skills

4:12

that I want all of my machines to have.

4:14

I also have a little bit of metadata

4:16

that I include here that makes it easier

4:18

for my agents when I'm running them in

4:20

this project to know which skills should

4:22

appear on which machines. So let's start

4:25

from the top. This was my agents MD in

4:28

Claude MD file that I have had in all of

4:30

my computers for approximately 2 years

4:33

now, give or take, maybe a bit less, and

4:35

I hadn't touched it cuz I just didn't

4:36

think it mattered that much. I also have

4:38

a video I did before about deleting your

4:40

Claude MD and agents MD. That was for

4:41

project Claude and agents MD files and

4:43

you're definitely writing those wrong. I

4:45

will show you what I mean in a bit. Just

4:47

we'll get there. Like, trust me. This is

4:49

my old file. I had global defaults.

4:51

First off, I called that repo specific

4:53

agent MD instructions override these.

4:55

User instructions override both.

4:56

TypeScript, I had one instruction. Avoid

4:58

any unless there is no reasonably typed

5:00

alternative or the user explicitly asks

5:02

for it cuz I was tired of agents just

5:03

writing any everywhere, especially like

5:05

GPT-4-1

5:07

and like Sonnet-4-4-2.

5:10

Really bad about all that. And then I

5:11

had some instructions here for commands.

5:13

I was really tired of agents spinning up

5:15

dev servers and breaking my environment

5:16

so I told them to stop. I said prefer

5:18

targeted verification first. Like, type

5:20

check and lint and focus tests. Run full

5:22

builds, yada yada. Package manager

5:24

preferences. I said prefer bun than

5:26

pnpm. Tech preferences, just the things

5:28

I want to use. Code styles, okay, I did

5:30

make one change here recently. I added

5:32

this YAGNI bit, which did actually

5:34

meaningfully help.

5:36

I don't know where this came from.

5:37

Another agent might have added this. And

5:39

then the computer inventory, this was

5:40

definitely added by Soul when I was

5:42

working on the fleet at some point. But

5:44

this was is a great agents MD. And I

5:46

just kind of lived with it for a long

5:49

ass time. Here is my new global

5:51

agentsmd. This is probably the least

5:53

important of the files, but we'll go

5:55

through the rest. I'm Theo, you're my

5:57

agent. We're working together a lot, so

5:58

I thought it'd be worth introducing

5:59

myself. Know from my YouTube channel and

6:01

work on T3 products like T3 GG, T3 Chat,

6:03

T3 Code, Uploading Ping, etc. I love to

6:05

build. I focus on building complex

6:07

things as simple as possible. I love to

6:08

find ways to reduce complexity when

6:10

solving problems. This is a small piece,

6:13

but I found this helps a lot just cuz my

6:16

preferences were not often met by

6:18

agents, especially with certain

6:21

overeager models like 5 6 Soul and Opus

6:23

5. I noticed that my agents were a bit

6:26

too uh proactive and were a little quick

6:29

to start writing a lot of code when they

6:32

didn't necessarily need that much code.

6:34

This two-sentence addition here helped a

6:37

ton with that in particular. I specify

6:40

that I want to share some of my

6:41

preferences here so we can be more

6:42

aligned as we work together. You might

6:44

be questioning why I'm writing this way.

6:46

A lot of it is because the models are

6:49

good at tone matching. If you talk a

6:51

certain way to the model, the model's

6:53

more likely to talk that way back, which

6:55

is something I very much wanted. You'll

6:57

see more specific examples of this in

6:59

the not-too-distant future. I do have

7:01

bullet point lists where they make

7:02

sense, like in my general coding

7:03

preferences. Say keep things simple.

7:06

Type safety is useful. Don't be scared

7:07

to propose bold ideas that can

7:08

meaningfully benefit our work. Be

7:09

careful with destructive actions that

7:11

are not explicitly requested. Tests are

7:13

good, endless smoke tests, regression

7:15

tests for features for feature

7:16

deletions, etc. Much less good. That

7:18

should be focused, not slop. Comments

7:20

are a great way to clarify functions and

7:21

how code is used. Don't comment every

7:23

line, but feel free to to describe

7:24

concisely how functions are used above

7:26

function definitions, classes, etc. And

7:28

keep comments up to date when making

7:29

changes. It's important to keep things

7:30

in sync. Coding preferences for

7:32

TypeScript, I have separate pieces here.

7:34

This is mostly cuz this is my agentsmd,

7:37

a lot of this is just to tune 5 6 Soul

7:39

to not suck as much at TypeScript.

7:41

Specifically, I said if your TypeScript

7:43

code looks like a Python dev wrote it,

7:44

it's bad TypeScript code. Avoid

7:45

one-liners that are just casting

7:47

wrappers. Write TypeScript in ways that

7:48

Matt Pocock and Theo would be proud of.

7:51

A very brief, if there's no tech for

7:52

these things in the repo, this is the

7:53

stuff I generally like. And when

7:55

building more complex web and React

7:56

Native apps, I like to pull in Zustand,

7:58

React Query, TanStack Start, Clerk, or

7:59

Better Auth if self-hosting, and

8:01

Archetype or Zod if purpose isn't an

8:02

issue. You get the idea. Questions are

8:04

read-only. This is another nice change

8:06

because I got annoyed at 5.5 and 5.6

8:09

being a little too quick to start making

8:10

changes when I ask about stuff in the

8:12

project. This helped a ton there. Match

8:15

ceremony to the task. Do not spawn

8:17

sub-agents or multi-agent panel for work

8:19

or a single agent finishes in one pass.

8:21

Delegation is for breath or adversarial

8:23

review, not for ordinary tasks. Several

8:25

agents do work in parallel. State file

8:26

ownership up front so they don't

8:27

collide. This again was noticing a lot

8:29

of sub-agents editing the same files and

8:31

having problems. This one line helped

8:32

quite a bit.

8:33

Visual and design work. Do not edit real

8:35

components first. Stand in contrasts.

8:38

Prefer dark mode with white texts. This

8:41

is me trying to get rid of the grays.

8:43

Glass radius. This is me trying to make

8:44

sure it doesn't destroy my production

8:45

environments. And then my pull request

8:47

section. This is a little aggressive

8:50

here, but I have a separate pull request

8:52

skill that has been very, very helpful.

8:55

I'm going to get this question a lot, so

8:57

I'm going to jump at it up front here.

8:59

Sorry if you answered already, but where

9:00

can I find the agents.md file? I want to

9:02

copy some of the instructions. You

9:03

can't. I very intentionally not publish

9:06

this file. Not cuz I'm evil and I want

9:08

you to have to carefully watch the video

9:09

and copy every single word by hand. The

9:12

real reason is the value of this isn't

9:14

the exact things I instructed. It is the

9:17

way I thought of it. It is the reasons I

9:20

added the things I added and the path I

9:23

took there. I'm still going over the

9:25

files and will do this for a bit, but I

9:26

will describe how I came to these

9:28

conclusions and why I added these things

9:30

right after.

9:31

I really want you guys to take the

9:33

opportunity to learn from my process

9:36

here,

9:37

not to copy-paste these specific things

9:39

in my global files. Copying all these

9:42

files would be like copying someone's

9:44

generic code template then using it for

9:46

every single thing you touch, but also

9:48

all of the things that you've worked on

9:50

before as well. These files influence

9:52

how all of your agents work across

9:54

everything you do.

9:56

You should control this file. You should

9:58

edit this file. You should think about

10:00

its contents and adjust it over time

10:02

based on your needs and your problems.

10:05

Anyways, let's look at my skills a bit.

10:07

I noticed myself saying the same couple

10:09

things to my agents over and over, and

10:12

they would often behave differently, and

10:14

they would often behave in ways that I

10:15

didn't love. In particular, when I asked

10:17

if my agents to babysit a PR or to file

10:20

a PR, the quality I would get out was

10:23

mixed.

10:24

So, I started originally with the

10:25

babysit PR skill, which has been very

10:27

helpful because a lot of my projects

10:29

have a lot of AI code reviewers on them,

10:32

thanks to the incredible channel

10:33

sponsors we have around that. Some

10:35

projects like T3 Code have three or four

10:37

of them on the project. And the point of

10:39

the babysit skill is to have my agent

10:41

monitor the PR doing things like

10:43

rebasing and pulling in main when it

10:44

should so that the PR stays fresh,

10:47

things like reading those review

10:49

comments and addressing them when they

10:50

come in. Most importantly, looping until

10:52

the PR is green and everyone's approved.

10:55

As I told my agents to babysit more, I

10:57

noticed certain failure patterns that

10:59

frustrated me. Little things like the

11:01

agent commenting through my account but

11:03

not indicating it was an agent to big

11:05

things like the PR getting three times

11:07

larger than it should have been because

11:09

it would address every single comment

11:10

like it was mission critical even for

11:12

things that didn't matter as much. So,

11:14

the point of this was to fix those

11:15

things. All the repos we work in have

11:17

various AI review bots. They're helpful

11:19

even if they're not always right. If

11:20

your harness offers tools to monitor a

11:21

PR, use them so you can respond when

11:23

comments arrive. Otherwise, pull the PR

11:25

for new comments and checks. Only act on

11:27

checks and comments newer than the

11:28

latest push. Verify every bot finding

11:31

against the source before changing code,

11:33

fix real findings and CI failures,

11:35

distinguish repository failures from

11:36

infrastructure flakes, and reply with a

11:38

written reason what when dismissing

11:40

false positives. I also specify to keep

11:42

an eye on changes to main and rebase

11:43

when needed. If an overlapping PR makes

11:44

this one obsolete, stop monitoring,

11:46

report to the user, and ask before

11:47

closing unless closure was explicitly

11:49

authorized. If a review bot leaves

11:50

feedback you believe is not worth

11:52

addressing, reply and resolve the

11:53

comment. Format comments left on Theo's

11:55

behalf as such. Markdown, model slug,

11:58

responding on behalf of Theo. Actual

12:00

reply. This has been super helpful.

12:03

Screenshots and videos help as well. I

12:04

have a custom file upload skill that I

12:05

built for my agents that allows them to

12:07

upload arbitrary files to R2 with a

12:09

public URL they can use to embed things

12:11

in PRs because I noticed them doing

12:12

nasty [ __ ] to try and get a video in a

12:14

PR. This solved that. This is probably

12:16

the most important line in here, though.

12:17

Do not let review feedback expand the PR

12:20

beyond the user's original goal. Address

12:22

real shortcomings, but avoid scope

12:24

creep. This skill's been great. One of

12:26

the most important details is kind of

12:27

hidden. Now, let me do that so you can

12:29

see it better. The description. This is

12:31

a thing that most people get wrong and

12:34

agents get even more wrong. Your

12:36

description should not describe

12:38

necessarily what the skill does because

12:41

the description is always inserted even

12:43

if the skill's not being used. The point

12:45

of the description is to let the model

12:47

know when it should pull in the skill.

12:50

So, you should treat the description

12:51

less like an elaborate explanation of

12:53

all the things the skill does. Hell,

12:55

I've seen skills where the description

12:57

gives you everything that you need and

12:58

you don't even need to use the skill cuz

13:00

the description has all the details. The

13:02

description should have the magic

13:04

keywords to tell the model when to pull

13:07

the thing in. So, here, I could probably

13:10

delete this monitor a pull request

13:11

through review and CI entirely and have

13:13

it just be "Use when the user asks to

13:16

monitor, watch, or babysit a PR." The

13:18

point of this skill is to be pulled in

13:21

and give the model the context when it

13:23

needs these things. I I it's unintuitive

13:25

cuz it's called a description, but you

13:27

really should think of the description

13:29

of a skill as the trigger keywords

13:31

rather than as an actual description. As

13:34

such, you can guess how my file PR skill

13:36

is formatted. File a concise pull

13:38

request. Use when the user asks to file,

13:40

open, or create a PR. Originally, I had

13:42

the file PR and babysit PR skills as one

13:46

skill, but I noticed once I added the

13:48

keywords that it was good enough at

13:49

triggering that I didn't really need

13:51

that anymore. So, I made them separate.

13:52

Also, I often want one and not the

13:54

other, so having them separate was

13:55

really nice. So, now I have the file PR

13:57

skill that triggers whenever a user

13:59

requests filing, opening, or creating

14:00

PRs.

14:02

This skill has been a night-and-day

14:04

difference to my quality of life. Not

14:06

because my agents couldn't file PRs

14:08

before, they could, but my agents would

14:11

often file PRs in ways that I [ __ ]

14:13

hated.

14:15

And now I don't hate it as much. Here's

14:17

an example of a PR that was filed by one

14:18

of my agents before I made these

14:20

changes. The title is fix server, parse

14:23

CLI version and update preflight.

14:25

Summary, verify pinned runtimes against

14:27

the CLI's actual T3 version output, make

14:30

the self-updating processor or fake emit

14:31

the same format, and keep wrong version

14:34

coverage realistic. Can any of you guys

14:36

say in plain language what the point of

14:38

this PR was? I know I can't. I happen to

14:41

remember what I did here. The problem

14:43

was that I was getting errors in the

14:45

little log in T3 code that showed when

14:48

there was drift between the version on

14:49

your machine and the version on the

14:51

remote machine. And this is the PR that

14:53

was made to fix that. A lot of my PR

14:56

descriptions looked like this, and the

14:57

titles aren't helpful either, and that

14:59

was annoying me. So, I gave direct

15:00

concrete examples to my agent here.

15:03

Before filing, check whether a PR for

15:05

this branch already exists. Review the

15:06

diff locally against origin main to make

15:08

sure its contents match the goal. PR

15:10

titles usually become commit messages,

15:11

so follow the repository's title

15:12

conventions. Look at recently merged PRs

15:14

and git history for example. Prefer a

15:16

concise human-readable title that

15:17

explains why the changes matter.

15:20

Bad example here, perf server, negotiate

15:23

per message deflate on the websocket.

15:25

Good example, perf server, cut websocket

15:27

frame size by 70% with gzipping. This

15:29

was a real example that I did cuz I had

15:31

a really awful title I hated from one of

15:33

my agents and gave it this example.

15:37

Agents are really good at bad and good

15:38

examples. So, if you notice your agent's

15:40

doing something poorly that you have to

15:42

go adjust, put one or two of those

15:45

inside of a skill or your globals, and

15:47

now the agent knows what bad and good is

15:50

to you, and you've seeded its weights

15:53

with the things that matter to you.

15:54

Another important chunk that I have

15:56

here, this is probably the one that's

15:57

changed my life the most as silly as it

15:58

is. Open the description with a simple

16:00

explanation of the problem based on the

16:02

user's original prompt, then briefly

16:04

explain the solution. Do not lead with

16:06

an implementation inventory.

16:08

So, I gave a bad example here, removed

16:11

implicit workspace carryover from every

16:13

new thread entry point. New threads

16:15

inherit only the project from context,

16:17

branch work tree audio yada. Just

16:19

nonsense, no one's going to read this.

16:20

Good example,

16:22

my new work tree default was ignored

16:23

when starting new threads on existing

16:25

work trees, super unintuitive. Now your

16:27

preference is always apply. So much

16:29

better.

16:30

I also specifically told it to stop

16:31

opening draft PRs because Codex loves

16:34

opening draft PRs that don't get

16:35

reviews. So, I told it to open real PR

16:37

rather than a draft so the review bots

16:38

run. If the user also asked to babysit,

16:40

continue with the babysit PR skill. I

16:42

probably don't even need this anymore

16:43

cuz it pulls the skills in correctly.

16:45

And you can see the difference if we

16:47

scroll up a bit and click any of my new

16:49

PRs. Here's a simple example, fix for

16:52

release, skip scripts during Vercel

16:54

installs. The hosted web deployment runs

16:56

its filtered Vercel install command

16:57

twice. Both installs invoke the root

16:59

prepare life cycle so the persistent

17:01

build cache accumulated effect TS go

17:03

backups until it hit the 100 file limit

17:05

and blocked releases. So, now we're

17:06

running with ignore scripts. That's a

17:08

lot clearer. I also have at the bottom

17:10

here in my example description,

17:13

maybe I don't have it here, but I have

17:15

it in my global. I think I have it in my

17:16

global.

17:17

I do. I should add it here as well. Add

17:19

a blurb to the end of the PR description

17:21

about what model and harness is making

17:23

the changes. This has been really

17:25

helpful for keeping track of what I'm

17:26

using for different work. Sadly, Codex

17:29

often doesn't know what model it's

17:30

using. I've been yelling at the Codex

17:32

team for a bit about this. Claude always

17:33

knows and most other harnesses know as

17:35

well.

17:36

Codex does not include the model slug

17:38

inside of the system prompt, which is

17:40

dumb of them. So, sometimes it'll say

17:41

that 54 filed it when it didn't, which

17:43

is annoying. But, yeah. Small thing.

17:46

Helps a lot to like get this information

17:47

in my PRs. I've been really happy with

17:49

this change, much more so than I

17:51

expected. Another simple example, I had

17:53

a bug in stash prompts where they were

17:55

being stashed per harness, which didn't

17:57

make sense cuz if I stashed a prompt for

17:59

Codex, I want to go throw it in Claude,

18:00

it annoyed me. I told it that, told it

18:02

needs fixing, and it described it

18:04

accordingly. Stashing a prompt,

18:06

switching providers, and restoring it

18:07

didn't work. The stash was bucketed per

18:09

provider instance, so after a switch,

18:11

the badge disappeared and the stash

18:12

looked lost. Restoring also dragged

18:15

dragged the stash model selection back,

18:16

defeating the point of moving a prompt

18:18

to a different provider. This is a great

18:19

description of why I cared enough to do

18:21

this thing. Super nice. Everything so

18:24

far has been focused on my global skills

18:26

in my global agents in Claude.md files,

18:29

but there's a lot more to this than

18:31

that. I want to show how I tuned the

18:34

files that we have in T3 code itself for

18:36

people contributing to T3 code. This

18:38

isn't for T3 code users, it's for T3

18:40

code contributors. It's much more

18:42

specific to the code base, which means

18:44

it would be helpful to figure out how I

18:46

discovered these things in the first

18:48

place. This was not a great example cuz

18:49

I actually explicitly told the agent to

18:51

not make the PR, but instead generate

18:53

the one-click link so I could make the

18:55

PR, and then I opened it and went and

18:56

tidied all of these things up. Yeah,

18:58

from here down is AI slop. I did include

19:00

that. This is the real thread I used to

19:03

do a lot of the changes that I'm talking

19:05

about here. To be clear, when you look

19:07

at the generated agents.md and

19:08

Claude.md, don't assume I used those cuz

19:10

I didn't. I was using them largely as a

19:12

reference point to get things tuned how

19:14

I wanted. I opened with a relatively

19:17

wordy prompt on what my goals were here,

19:20

that I want to rethink the agents MD and

19:21

Claude MD for T3 code. I want them to

19:23

better match the tone of my global

19:24

agents and Claude MD. I also want to

19:26

better represent the project and how

19:27

agents should work. Random additional

19:29

things I want to touch on. These are

19:30

things I knew had been happening a lot

19:31

that were annoying me. And also I wanted

19:34

to pull in my glossary from my lakebed

19:36

agents MD cuz I found that that has been

19:37

super super helpful for how I

19:39

communicate with the agent. I also said

19:41

to match the tone from that file. Write

19:43

up two separate drafts for an

19:44

alternative Claude MD based on what I

19:45

shared here. And it wrote them. And I

19:47

didn't like them much. There were parts

19:48

there that were good, but it wasn't

19:49

great. But I asked it to do something

19:51

that ended up being significantly more

19:53

helpful. Can you look through my history

19:55

with models like Fable, Opus, and

19:57

GPT-5-6-Soul

19:59

in Claude code and Codex on this machine

20:01

to see what the most common mistakes

20:03

are. Want to make sure we optimize to

20:05

steer away from those. I want you to

20:08

break down the most common failure modes

20:10

and how often each model hits them. This

20:12

was fun because not only did it catch

20:15

specific failures that I cared about, it

20:18

also categorized them by how often they

20:20

happened with each model and harness.

20:23

I noticed Opus 5 was aggressively

20:24

killing the wrong process, often killing

20:26

the T3 code instance that it was running

20:28

in. And it now is confirmed with real

20:31

numbers. Even though I only used it for

20:32

2 days, it had done significantly more

20:35

times to kill processes than Fable had

20:36

in all the time I used it. Here you can

20:38

see all the times user data were

20:39

touched. Thankfully it was almost all

20:41

read-only inspection, but we should

20:43

still audit it further.

20:44

PR hygiene, we noticed Soul, see again,

20:47

40% of the time it filed PRs it was

20:49

filing drafts. Other models were much

20:51

less likely to file a draft. A lot of

20:53

repo-wide check failures where it was

20:54

running things it didn't need to, taking

20:56

way more resource and build time on my

20:58

machine. Good call out that it had here.

21:00

And it also tracked the bash error rates

21:03

for the different agents that were

21:04

working here here well. I will say that

21:06

I did all of this with Opus because I

21:08

was out of Fable at the time and it

21:09

didn't do quite as good of a job

21:11

auditing as both Fable and 5 6 did. I

21:14

did have all of them do audits looking

21:16

for various failures and try to collect

21:19

as many of them as possible so that I

21:22

could use that as a reference point. As

21:24

I mentioned, I had a lot of agents do

21:26

this audit with me and find all the

21:28

weird failures that happened across all

21:30

of my real runs on my real machines. I

21:34

forgot which model made this one, but I

21:35

really really liked it.

21:38

This was measuring corrections per 100

21:40

user messages per model. So, I would

21:43

often tell Fable it implemented things

21:45

wrong, but that's also cuz I use Fable

21:46

to do the hardest stuff and I often give

21:49

Fable less context going in. I also

21:51

apparently tell Opus 5 that its results

21:53

look wrong way more often. It looks like

21:55

Soul's really good here, but remember I

21:57

don't use GPT models for UI stuff. So,

22:00

if I say it looks wrong, I'm probably

22:02

just going to move off of Soul. For

22:04

process, I actually had more problems

22:06

with Fable than other models for like

22:08

leaving branches stale, not fixing CI,

22:10

stuff like that. Opus 4 8 would misread

22:14

what I was asking for very often. Tool

22:16

misuse, Opus was the worst by far. Opus

22:19

5 was misusing [ __ ] all the time and

22:21

breaking my environment. Overbuild, Opus

22:23

5 and Opus 4 8 were the most aggressive

22:26

with. Apparently, Fable's slightly worse

22:28

than Soul overall, but that's a

22:29

universal problem. Stopping early was

22:31

also universal problem. No verify was

22:33

pretty universal. Regression was rare,

22:35

but expensive when it happened. And only

22:37

Soul was doing unasked edits that I had

22:39

to tell it like, "Hey, don't edit that.

22:40

Undo that." And then it proposed a bunch

22:43

of changes to my global files that it

22:45

thought made sense. And I sat and read

22:48

through this. And this one. And this

22:51

one. And even this one.

22:53

The point of doing all of this is that I

22:55

wanted to get a better feel for all of

22:58

the things that went right and wrong in

23:00

my history. Some of this is just vibing

23:03

it out and gut feel based on my own uses

23:06

and my own work in these projects, but

23:09

it's also a lot easier to go through

23:11

your history with an agent or dozens of

23:13

them in this case, than it is to read

23:16

through all of that yourself and really

23:17

think about it. And getting stats on

23:19

what the failure cases are and getting

23:21

your agents to give you the feedback on

23:23

what happened is super helpful. And if

23:25

you notice certain threads that just

23:27

didn't go how you want, you should work

23:30

with the agent to fix that. You should

23:32

ask it, "Why did you make this decision?

23:34

What gave you indication that this was

23:36

the right direction?" It might be

23:37

something that's outdated in your Claude

23:39

MD or agents MD. It might be something

23:41

that it read incorrectly earlier and

23:43

then followed along once it was in

23:45

context that didn't actually make sense.

23:47

Think about these failure modes, notice

23:49

them when they happen, and ask the agent

23:51

questions. Another really common one is

23:53

if you ask for what you think will be a

23:55

simple change and the agent takes 30

23:57

minutes or more when you thought it

23:59

would take five. Ask the agent why it

24:01

took longer. What did it spend its time

24:03

on? Ask it to categorize all of the tool

24:05

calls it made into different groups and

24:07

try to determine which ones were and

24:09

weren't helpful so that you can make

24:11

adjustments to help the model make

24:13

better decisions in the future. And I

24:14

guess with that, there's no better time

24:16

than now to dive into our agents MD

24:18

file. Another very important thing is

24:20

that your agent MD should be incredibly

24:23

different from your read me MD. The

24:25

point of a read me is context on the

24:27

project and what it is that is useful

24:30

for humans and agents that are trying to

24:32

decide if they want to pull in this

24:33

code. The point of the agent and Claude

24:35

MD files is to tell the agent how to

24:38

change things in the code base and what

24:40

it should know before it makes those

24:42

changes. So yes, there are parts of this

24:44

that should be descriptions of the

24:46

project, but the majority of what's

24:48

going on here really should be focused

24:50

on how your agents will work in the

24:53

project. While there is a lot of overlap

24:55

between developers contributing to a

24:56

project and agents doing those

24:58

contributions, this file really should

25:00

not be written for devs, it should be

25:01

written for agents to help the agent

25:03

interact with the dev better. And with

25:05

that said, one of the most important

25:06

details you can get right in these is a

25:08

glossary. I like writing these not just

25:11

so the agent understands me, because

25:12

it's usually pretty good at that. It can

25:14

infer what I mean when I talk about a

25:18

thing that I want it to do. The glossary

25:20

can help it understand what I'm saying a

25:22

little bit, but the glossary's much more

25:24

helpful for the agent to describe things

25:26

to me the way I want it to. And a lot of

25:29

these base terms are really simple. Most

25:31

of them are terms that it probably

25:32

already knew, but when you give it this

25:35

type of really simple language to

25:37

describe things with, it becomes much

25:39

easier to talk to the agent because it

25:41

knows how you're thinking and how you

25:43

and your team are working. So, our

25:45

glossary's really simple here. The first

25:47

word defined in the glossary is you,

25:49

which means the agent reading this file

25:51

and changing T3 Code. We us and

25:53

maintainers mean Theo Julius and the

25:55

people building T3 Code. These are who

25:57

you're talking to now. User means the

26:00

person using T3 Code to direct coding

26:02

agents. An agent means the coding agent

26:05

a user runs inside of T3 Code. Depending

26:08

on context, this may also include you.

26:10

Provider means the agent runtime or

26:12

harness that T3 Code talks to, such as

26:14

Codex, Claude, Cursor, or Open Code.

26:17

Client means the web, desktop, and

26:18

mobile UIs. Environment means one

26:20

running T3 server and the machine, file

26:22

system, provider credentials, and state

26:24

that it owns. A project is an

26:26

environment local workspace record

26:27

rooted at a directory. You get the idea.

26:30

These are all very simple terms. I think

26:32

this is easier from the top though, so

26:34

let's do that. T3 Code's a minimal GUI

26:36

for coding agents. A node web socket

26:37

server wraps provider CLIs like Codex,

26:39

Claude, Code, Cursor, Grok, and Open

26:41

Code, and it serves web, desktop, and

26:42

mobile clients. You can think of T3 Code

26:44

as an open source bring your own

26:46

subscription alternative to apps like

26:47

Claude Desktop, Codex App, Cursor Glass,

26:49

and Conductor.

26:51

This is useful context because now the

26:52

agent has to spend less time figuring

26:54

out what it is or what to look at when

26:57

it's trying to model UI and UX. Now it

26:59

has very specific direct instructions.

27:02

Now it knows what to think about when

27:04

it's trying to get things right. And

27:06

then an important piece, what makes T3

27:07

code special? Again, you might think,

27:09

"Oh, that's something that should go in

27:10

the read me." Yeah, to an extent. But

27:13

the more important part of putting this

27:14

here is to give the model an idea of

27:16

what it should never compromise on. If

27:18

it's making changes and those changes

27:20

hurt any of these things, it shouldn't

27:22

make those changes. It also helps

27:24

contextualize how it's working and where

27:26

it's working, so it knows things like

27:28

since I say we're open at the core, that

27:30

means this is an open source project.

27:31

Act accordingly. Or things like we have

27:33

over 100,000 users who love T3 code,

27:35

that means that despite being alpha, we

27:37

have to be careful to not break things.

27:39

I even say here that it's important we

27:40

maintain the things that our users love

27:42

as we continue to iterate on the

27:43

product. Here's a brief list of the

27:45

things we can never compromise on. Open

27:47

at the core. T3 code is truly open. We

27:49

share our road map, we share how we

27:50

think about things, and of course we

27:52

share all of our code. A large number of

27:53

our users run forks. We work in the open

27:55

and we should strive to stay that way.

27:57

Two is performance without compromise.

27:58

Lots of apps have gotten bogged down

28:00

with bad tech decisions and slop. We

28:01

have not and we're proud of the

28:02

performance of T3 code. We regularly

28:04

audit for performance regressions, often

28:06

caused by sending too much data over web

28:07

sockets, CSS animations causing GPU

28:09

spikes, lists being hard to render, and

28:11

more. Make sure all changes are

28:13

considerate of performance impact.

28:15

Three, remote ready. The architecture of

28:17

T3 code's web socket layer with NPX T3

28:19

enables a lot of awesome remote

28:20

features. These have become core to the

28:22

product. Whether users are connecting

28:24

directly over their local network using

28:26

Tailscale or leaning in fully with T3

28:28

Connect, which is our tunnel solution

28:30

also in this repo, we need to make sure

28:32

new features are properly supported. And

28:34

then I have the multi-surface callout

28:36

because T3 code has its three core

28:37

surfaces, web, desktop, and mobile. Web

28:39

is kind of two services. We have the

28:41

public-facing app T3 code's as well as

28:43

the locally hosting web app through NPX

28:45

T3 command. Both need to be supported by

28:47

all new features where reasonable.

28:49

Desktop is the main surface that most

28:50

users install first. It's a full

28:51

electron app that bundles the server

28:52

runner as well. The desktop app can also

28:55

be used as the host server allowing

28:56

remote connections from both app T3

28:58

codes or the mobile app. And there's

29:00

mobile which is a react native app for

29:01

both iOS and Android. Mobile app allows

29:03

for connecting to any T3 code server to

29:04

control work remotely. Still in early

29:06

access test flight but it's pretty close

29:08

to shipping globally. This I need to go

29:10

remove now because we launched the app.

29:12

This is all of the context that I want

29:13

the model to have going in about what T3

29:16

code is so that I can make better

29:18

decisions and not spend its first four

29:19

tool calls googling and reading through

29:21

files to figure out what the [ __ ] we're

29:22

doing. I then do my favorite, a note

29:25

from Theo. I like ambitious ideas,

29:26

simple systems, and software that feels

29:28

obvious. Do not preserve complexity just

29:30

because it already exists. Do not

29:31

introduce machinery because it looks

29:33

architecturally impressive. Understand

29:35

the real constraint then fight for the

29:37

smallest model that makes the correct

29:39

behavior unsurprising. Channel both

29:41

measure twice cut once and YAGNI. These

29:43

are just phrases in the dev world that

29:46

are associated with smaller concise

29:49

changes and not adding things that are

29:50

unneeded. Fight scope creep. Try to

29:52

honor the dev's intent in both a minimal

29:54

and realistic fashion. The rest of this

29:56

document is meant to help you navigate

29:58

the code base and make changes

29:59

effectively. Think of these instructions

30:01

less as hard rules more as good

30:03

defaults. This is another important

30:04

thing because I've noticed sometimes

30:06

I'll put things in here that a user

30:07

doesn't want and then they get really

30:09

annoyed if the agent and the user

30:11

disagree. So I always put something like

30:13

this in in order to make sure the person

30:15

prompting can get around the stuff I put

30:17

here if it doesn't make sense. I even

30:19

specify that developer preferences

30:20

should be able to override anything

30:22

here. Of note, most T3 code

30:25

contributions will come from T3 code

30:26

itself often controlled remotely. This

30:29

means you should be careful about

30:30

accessing data, killing dev servers, and

30:32

other things that may damage the T3 code

30:34

instance that the contributor is using.

30:36

This is because I got really annoyed at

30:37

Opus killing my running jobs. It was so

30:41

[ __ ] annoying that it inspired me to

30:42

do this whole rewrite. So, I guess Opus

30:45

has contributed a lot. If you haven't

30:46

caught up on the podcast, Opus 5 went

30:48

from my favorite model to one that

30:49

pisses me off a ton really fast.

30:51

I don't use it if I can avoid it at this

30:52

point, but yeah. This was because Opus

30:55

was pissing me off. And then we have the

30:56

glossary, and then I have the three ways

30:58

to hurt yourself. This came pretty much

31:00

directly from Soul auditing my logs and

31:02

finding these particular failure modes

31:04

that happened a lot. I have a few more

31:06

pro tips coming up for real-world work,

31:08

as well as two skills that have changed

31:10

my life that I've never shared before,

31:12

and I'm really excited to show you guys

31:14

all of that. But this video is getting

31:15

long, so I hope you can forgive one more

31:17

quick sponsor break. If you're a

31:18

developer and you watch my videos, or

31:20

you've talked to me in person, then you

31:21

probably already know how much I love

31:23

today's sponsor. It's Convex, and I

31:25

don't normally ask them to pay me

31:26

because I shill them for free. Their

31:28

stuff is so good that we built all of T3

31:30

chat on top of it after my own

31:31

self-rolled solution just wasn't cutting

31:33

it. And I'm so thankful I did, because

31:35

Convex solves so many annoying problems.

31:38

They want me to show their new CloudKit

31:39

plugin, and I get it. It makes the model

31:40

better at Convex, but it doesn't really

31:42

need it because the models already are

31:44

all so good at Convex. It turns out

31:46

having one folder in your codebase that

31:47

represents all of the things that your

31:49

backend, database, etc. all need makes

31:51

it really easy for agents to write code

31:53

against it. This is why I love Convex.

31:55

It's just code, and it's just TypeScript

31:57

code, too, which makes it really safe.

31:59

And if you're worried about scale, don't

32:00

be. Jamie and James are two of the most

32:02

talented engineers I have ever met. They

32:04

helped Dropbox scale. I've never met

32:06

somebody who can tell you all the flaws

32:08

of S3 scalability quite like these two

32:10

can. The real reason to use Convex is

32:12

all of the annoying problems it solves.

32:14

From keeping data synced to preview PRs

32:16

to access controls to just all of these

32:18

things that normally take a lot of work

32:19

for you and your agents, Convex just

32:22

makes it easy. Build apps in a way that

32:24

you and your agents will love at

32:25

swade.link/convex.

32:26

Sorry about that. I hope you understand

32:28

writing markdowns expensive. And writing

32:30

it all by hand is

32:31

time-consuming. A bit of this was

32:33

Whisper flowed, but the majority of this

32:34

was hand-coded and written, so yeah.

32:37

This next piece is another one of those

32:38

huge lifesaver ones because I was

32:41

running into a lot of problems where I

32:42

would make a change in T3 code and it

32:44

would only apply to one of the surfaces.

32:46

For example, it would only be changed on

32:48

the desktop app and not the website or

32:50

mobile. I wanted to stop doing that and

32:53

also to make sure the docs kept up to

32:54

date as well. So, I said as much. Hit

32:57

every surface. The most common defect in

32:59

this repo is a change that works on the

33:01

path you tested and is missing

33:02

everywhere else. Before calling

33:04

front-end work done, walk this list and

33:06

see which entries apply. Make sure all

33:09

the entry points are addressed, things

33:10

like settings page, command palette, and

33:12

keybindings. So, if you add a feature in

33:14

the chat view, it should be accessible

33:15

in those places. You have clients, cuz

33:17

we need to make sure things work in all

33:18

the different clients, the web version,

33:19

the desktop version, as well as the

33:21

mobile apps. Shared logic lives in the

33:23

shared package client runtime section.

33:26

And then the providers. Each of our

33:27

providers have their own adapters.

33:29

Provider ship features need a decision

33:31

per adapter, even if the decision is not

33:32

supported here. Very helpful, especially

33:35

when we add features for Codex and

33:37

Claude. Now, you have to decide, okay,

33:40

not Cursor, Grok, or Open Code because

33:41

their implementations aren't quite as

33:43

far along, or maybe you do decide to

33:44

sneak in Grok cuz their SDK has made a

33:46

lot of progress. Super helpful. We also

33:48

have this contracts package which

33:50

describes everything that crosses the

33:52

wire over the websocket server or the

33:54

new HTTP backed like cold store. A lot

33:57

of the time changes will be made in the

33:58

app that aren't properly reflected in

34:00

the contracts package. This one line has

34:02

fully fixed that. This was another

34:04

really big one, reverse states. I had a

34:06

lot of instances where I added things

34:07

like settle and snooze, but the logic to

34:10

unsettle or unsnooze was not provided.

34:13

This helped a ton with that. Same with

34:16

the connection modes, telling it the

34:17

ways it can connect, make sure that they

34:19

all work as expected, and then the docs.

34:21

This callout ended up making me go even

34:23

further, and I believe I still have a PR

34:25

open for this. I do. I made a PR that I

34:29

have to update and get merged soon where

34:30

I properly split user and maintainer

34:32

docs, so they're two entirely separate

34:34

entry points, because I've had a lot of

34:36

problems with models writing a bunch of

34:38

the internal details of how something

34:40

works in the user-facing docs. I had

34:42

this a lot with Lakebed, and I still

34:44

have it a bit with T3 Code, and I found

34:46

that a really clear split, plus a little

34:48

bit of tuning in your agent MD to steer

34:50

the model in the right direction, helps

34:52

a ton with this. Also give some context

34:54

on dev servers, because I noticed the

34:55

model often trying to call bun, even

34:57

though the whole project is Vite Plus

34:59

now, and it's called out everywhere.

35:01

Models are also really bad at Vite Plus

35:03

without a little bit of help, because

35:04

they'll try to type out Vite Plus, and

35:05

it doesn't work. They have to use VP,

35:07

and they'll also use VP dev instead of

35:09

VP run dev, which is really annoying. I

35:11

called out Evan You for this decision

35:14

already, where VP dev does its own

35:16

thing, and VP run dev does what you

35:19

actually want, even though PNPM dev and

35:22

bun dev do the obvious thing. VP dev is

35:24

special, and that pissed me off a lot at

35:27

the time, because it annoyed me to have

35:28

to remember to type the right thing.

35:30

Then my agent started making the same

35:31

mistake, and now I'm even more pissed.

35:33

So, this is a line that exists because

35:34

our friend Evan You could not get over

35:36

his [ __ ] It is what it is. He's

35:38

wrong. It is hurting real users. Now I

35:41

have to add this line and burn millions

35:43

of tokens over the next few years,

35:44

because one person thought this was a

35:46

good idea. Ugh. Other pieces in here

35:48

that were helpful is I gave instructions

35:50

for using custom T3 Code home

35:52

directories, so that if you spin up a

35:53

dev server, it doesn't break your

35:55

running server, as well as the ability

35:57

to share a dev server over Tailnet and

35:59

Tailscale, which has been awesome for

36:01

doing a lot of remote dev work. I also

36:03

noticed it was often sharing the wrong

36:04

URL without the pairing like token that

36:07

I needed, so I told it to do that. And I

36:09

also noticed that it was stopping the

36:10

wrong processes, and often forgetting to

36:12

stop the one that it should, so I told

36:13

it to make sure it saves the PID when it

36:16

spins up the dev instance, so it could

36:17

kill it. I gave explicit instructions

36:19

for test data, which have been very

36:20

helpful as well,

36:21

as well as some instructions for

36:22

verifying. I got annoyed at it spinning

36:24

up computer use when it didn't need to,

36:26

So, I specified here "Upon request,

36:28

user-visible front-end changes should

36:30

get one integrated pass in a real

36:31

client." And we have skills for this,

36:33

test T3 app and test T3 mobile that

36:35

Julius wrote for spinning up and testing

36:37

those things with computer use. Then a

36:38

bit more pull request instructions so

36:40

the people filing pull requests will get

36:43

better stuff when they do it from their

36:45

agent cuz a lot of We have like 500 PRs.

36:47

I want everything I kind of make them

36:48

easier to read. This has been very

36:50

helpful already with that. The where

36:51

code lives probably isn't that necessary

36:53

anymore, but it's so little whatever,

36:55

it's fine. Little taste section about

36:57

where we want things to live. Complexity

37:00

belongs at the adapter boundary,

37:01

orchestration stays pure, UI stays dumb.

37:03

Inferred types over annotations, any is

37:05

the enemy, yep. Comment subscribe how a

37:07

thing is used and move with when the

37:09

code moves to be used mostly to describe

37:11

functions, not to annotate every line of

37:12

behavior. Our users drive agents all day

37:14

and notice drop frames, lying spinners,

37:16

and stale light labels. No continuously

37:18

repainted animations, they peg the GPU.

37:20

For real here, fight the task in front

37:22

of you, say so loudly, and get a human

37:23

sign-off before breaking it. One last

37:25

one is I noticed that the dev servers

37:27

were being spun up and worked on with

37:30

way too much security for something that

37:32

only affects your local network and

37:33

tailnet. So, I yelled at it for that and

37:36

said security's important, but you not

37:37

be over-indexed on, especially for dev

37:39

mode and maintainer-only features.

37:41

Another one of those additions that

37:42

seems small, but has actually been

37:44

life-changing for a lot of this work.

37:46

The harsh reality is that you don't

37:47

notice complexity that isn't necessary

37:49

until it crosses a certain threshold.

37:51

So, the problem I had is that stuff was

37:53

getting unnecessarily complex. I only

37:54

noticed when it was too far, but then I

37:57

took all of the things that caused that,

37:58

I put them here, and now everything is

38:01

less complex, which has been

38:03

awesome. It's been so nice to have

38:06

everything I do just be more like what I

38:09

want, and my prompts have gotten way

38:11

smaller as a result.

38:13

Here's a real example.

38:15

I have to add the iOS and Android apps

38:17

for T3 code to the marketing site. We

38:19

are working on a marketing site

38:20

overhaul, but I want these ASAP. So,

38:22

we're going to get them right now. I

38:24

asked it to make these changes. It

38:27

decided that it wanted to give me a mock

38:29

for all the different options using one

38:30

of those two skills I'll show you in a

38:32

minute. And it created these mocks.

38:34

A is marketing hero with the also on

38:36

your phone call out. B marketing final

38:38

call to action. C download page. D

38:41

marketing footer.

38:43

I hate most of these.

38:45

It wants C plus D plus A together.

38:47

Do C plus D plus A.

38:50

File and babysit. That is my whole

38:52

prompt.

38:53

That prompt will do everything I want

38:56

the way I want it to. The C plus D plus

38:58

A, it knows because that's the context

39:00

it gave me using my HTML scale. And file

39:03

and PR tells it use these skills to

39:06

figure out the rest that I want. Here's

39:09

one I did yesterday that I forgot to

39:10

make a PR for where I'm trying to fix

39:12

the remote upgrade path to just have

39:14

better UI.

39:16

File and PR.

39:17

And now that will be filed and I'll

39:19

probably go merge it after I'm done

39:20

recording.

39:21

So, I just noticed a bug literally now

39:23

where I had a slept thread that woke cuz

39:26

I snoozed it and it unsnoozed, but I

39:28

wanted to settle it and the settle

39:29

button wasn't working on a thread that

39:30

had just woken up. So, I described the

39:32

bug. I had a woke thread that I wanted

39:34

to settle and I was unable. The settle

39:36

button that appeared on hover would not

39:37

work. Right click settle did. Diagnose

39:40

and fix, file and babysit. That is

39:42

everything I need and I'll have a PR

39:44

that is probably ready to merge in the

39:46

next 15 or so minutes. That's pretty

39:48

[ __ ] cool. I know it's silly, but

39:50

this has been really nice.

39:52

But I promised you guys more.

39:54

Promised you guys skills I hadn't shown.

39:56

So, I'm going to do that now. The first

39:58

ones I want to show here are my command

40:01

center skills. These are the ones that

40:02

are specific to just this machine

40:05

because I bring new computers into my

40:06

network that I want to use for different

40:08

things all the time. I do need to get in

40:10

the habit of sunsetting a bunch of them,

40:12

but I'm not there yet. But when I get a

40:14

new Linux box set up where I just

40:16

re-image a machine, I don't have to put

40:18

a bunch of work in or set up Nix or some

40:20

[ __ ] I just want to get that machine

40:22

online ASAP. So, I ended up configuring

40:25

a bunch of things for my machine,

40:27

getting my laptop how I wanted, and then

40:29

telling an agent to look at all of my

40:31

config, look at my bash history, look at

40:33

all the things I set up, and write them

40:35

all down. And then once I did that, I

40:37

told it to go set up another machine

40:39

with all of the things I liked. And it

40:41

got a lot of it, but it missed some

40:42

things and it did some things it didn't

40:43

need to. So, I told it all of those. I

40:45

told it to adjust those instructions to

40:48

better fit the needs that I had. And it

40:50

did.

40:51

It created a whole provision a box skill

40:54

that is all of the things that I want to

40:57

have on these new machines that I can

40:59

set up over SSH. I even got it to make

41:01

me this nice little HTML page that it

41:03

updates every time it makes changes to

41:05

the fleet that detail all the machines,

41:07

their specs, what they do, what exists

41:10

on them, etc. Different connection

41:12

methods,

41:14

the different specs,

41:16

the baseline for what I want on each

41:18

box, tmux, certain shell stuff, the

41:21

ability to control tmux over SSH with a

41:23

mouse. And as you've probably seen in

41:25

previous videos, that's exactly what I

41:26

have set up. I have this machine. This

41:29

is the one that does most of my dev

41:30

work. It's my Framework Desktop with 32

41:32

threads and 64 gigs of RAM. This machine

41:35

gets abused and it is where I do all my

41:38

dev work now, and it runs a T3 code

41:40

process that everything is controlled

41:42

through. All of my machines are

41:44

color-coded and also have matching

41:46

colors inside of my actual like tmux, so

41:49

it's easy to see each machine in SSH

41:51

when I need to. But honestly, since the

41:52

recent T3 code changes, I have been

41:54

doing a lot less. I'm just using

41:57

everything through T3 code and not

41:58

bothering with the SSH. You get the

42:00

point though. I set all of this up so

42:02

that I can more easily manage and

42:03

configure my machines, and I have a

42:05

skill that is only on this computer, but

42:07

this is in a repo that has all the

42:08

skills that all the other computers get.

42:11

So, let's look at the universal ones cuz

42:12

I promised you guys I would show some

42:13

skills I have never shown before. I

42:16

already mentioned the file upload one.

42:18

This needs a lot of tuning in the

42:20

description cuz a lot of this one I will

42:21

admit is slop. The whole service I built

42:24

is also slop. I actually bought

42:27

tslop.org in order to have a place on

42:29

Cloudflare that these files will go. And

42:31

this is my file upload skill for

42:33

uploading any local file {m-dash} a

42:36

screenshot, screen recording, log,

42:37

document, markdown file, config, build

42:39

artifact or archive to the user's public

42:41

file host at files.tslop.org and return

42:43

a public URL.

42:44

Rewriting all of this. When user asks to

42:46

upload

42:49

a file

42:50

or one is needed for a PR

42:53

PR descriptions

42:55

use this skill.

42:57

Cool. Much better description. I also

42:59

have a requires section I added here in

43:01

my metadata because I don't want the

43:03

skill on machines where I don't have a

43:04

global file host token cuz that is how

43:06

this skill gets authenticated.

43:08

If it is unset, tell the user instead of

43:10

guessing. And I have the simple bash

43:11

command that it will run in order to

43:13

post the file to my service. Needs the

43:15

token. As long as it has the token, it

43:16

can upload. So, you use only the file's

43:18

base name for file name such as

43:19

loginflow.mp4. The server slugifies it

43:21

and adds a random suffix, so names do

43:23

not need to be unique. Treat the

43:24

response body, yada yada yada. And then

43:26

basic instructions on how to use the URL

43:28

in GitHub as well as instructions that

43:29

it can encode a GIF for a preview when

43:32

it makes sense to.

43:33

And I even give the ffmpeg command for

43:35

that, which probably wasn't necessary.

43:37

This skill is [ __ ] awesome. If any

43:40

idea how cool it is to have agents that

43:42

can upload files. So, if you're on your

43:44

phone on the T3 Code mobile app and your

43:47

agent builds some new feature and you

43:48

want to see it, you can tell it upload

43:51

the file and respond with the URL. And

43:52

then it does, and then you can click the

43:54

URL and watch the actual video it

43:56

recorded on a real computer of it doing

43:58

the thing. You can tell it to file a PR

44:00

and it can include that URL in the

44:01

embedded video in the actual PR. It

44:04

makes the back and forth so much easier.

44:07

And this is the theme I really want to

44:08

make sure you guys get out of this. I'm

44:10

not just trying to make the model more

44:13

technically competent. The role of all

44:15

of this is not to make the model better

44:17

at writing code. There are little pieces

44:19

here that are helpful with that. But the

44:22

main point of all of this, the main

44:24

reason I put all this time in

44:26

is to make the model better at

44:28

communicating with me. I want to read

44:30

the outputs, but when the outputs are

44:32

piles of slop with no useful context, it

44:35

sucks. All of these skills, all of these

44:37

capabilities, all of this time I spent

44:39

is giving the model what it needs to

44:41

communicate with me and other

44:43

maintainers and contributors better. And

44:45

the file upload skill is not super

44:48

useful for the agent, but it's really

44:49

useful for me when I want the agent to

44:51

show me something. And that was all the

44:53

preamble for the skill that I rely on

44:55

the most, the one that has fundamentally

44:57

changed my life and sadly requires me to

44:59

shout out Anthropic because Thoric is

45:01

the guy who HTML pilled me. This is my

45:04

HTML planning skill. It's no longer HTML

45:06

planning, so I use it a lot for things

45:08

that aren't plans. It's now HTML

45:10

communication. When the user asks for an

45:13

HTML write-up of work, not as part of

45:15

the code base, use this skill to create

45:18

and always upload it to post plan. It's

45:20

also useful for reading post plan.dev

45:22

URLs cuz I noticed the agent would try

45:24

to browser use to go to post plan when

45:27

it can just curl it. So, when should

45:28

this be used? Use this skill for any

45:30

request to produce a readable HTML

45:32

artifact for a human, whether it's

45:34

called a plan, a spec, a write-up,

45:35

findings, a summary, a report, a

45:36

comparison, a set of UI mocks. The word

45:38

plan is often absent. You can tell I

45:40

wrote this by hand cuz of the typos

45:42

here. What the request share is a

45:45

document to read outside the terminal

45:46

and a link to open it. Horrible copy.

45:47

I'll touch this up later. Also, use it

45:49

whenever a user supplies a post plan

45:51

URL. A lot of this was slop I tidied up

45:53

incorrectly. Honestly, probably going to

45:54

delete almost all of this. But the rest

45:57

is what matters. Reading a post plan

45:58

URL. I want the model to stop opening

46:01

this in the browser directly. So, now I

46:03

say when a user supplies a post plan dev

46:05

URL, fetch the uploaded HTML immediately

46:07

with the shell. Do not use web search or

46:10

browsers to retrieve it. You know what

46:12

I'm going to do?

46:13

I'm going to show you guys how I

46:14

actually make changes here because I'm

46:16

seeing opportunities to improve. Real

46:19

prompt I just wrote that might be

46:20

helpful for you guys to see how I work

46:22

with agents. This is in my fleet repo

46:24

where I manage all of this content. I

46:26

want to make a few more small changes.

46:27

I've made more since my last prompt, so

46:28

refamiliarize yourself before editing.

46:30

The HTML communication skill is a bit

46:32

overloaded. I want to break it into

46:34

similar to the file PR and babysit PR

46:36

skills cuz I worked with this instance

46:38

in this thread for that in the past. The

46:40

descriptions for both should be really

46:41

simple, focused on when to trigger and

46:43

not what it is. One should be post plan

46:45

read for reading content from post plan

46:47

URLs. The other should be HTML

46:49

communication for communicating to the

46:51

user via HTML. When to use section

46:53

should not exist in the post plan skill,

46:55

only the HTML communication one. It

46:57

should also be more minimal.

47:00

Make your changes. Don't commit or push

47:02

to any machines yet. I'll tell you when.

47:04

This is just cuz I'm using soul and I

47:06

want to make sure it doesn't overdo, so

47:08

I tell it where to stop. When you give

47:10

the model a stop point, life gets much

47:12

better. Remember earlier when I said to

47:13

go fix the feature with settle, diagnose

47:16

and fix file and babysit? Apparently it

47:18

was a straightforward CSS bug fix. Okay,

47:20

there was a pointer events none that was

47:23

breaking or that should have existed

47:24

that didn't on the woke label that was

47:26

causing it to break the button that I

47:29

had. I don't love this comment, but [ __ ]

47:32

it, yolo.

47:34

Let's merge it.

47:36

Cool. And now that's gone. Man, I love

47:37

the new T3 code workflow so much.

47:39

While I was doing that, this finished

47:41

and now we have the changes and my new

47:43

skills. So, we can read these quickly.

47:46

Use this skill when the user wants a

47:47

plan spec write up finding summary

47:49

report comparison or set of UI mocks

47:50

presented as readable HTML. Do not use

47:52

it for HTML that ships as part of the

47:53

product. Yep. Write one self-contained

47:56

HTML file, capped at 512

47:58

write it like a spec, not a landing

47:59

page. Sure.

48:01

It's like basic formatting things that I

48:03

left in there.

48:04

UI mocks, label them A, B, and C for

48:07

easy selection, lay them out for direct

48:09

comparison, keep one file across

48:11

iterations, so it's post plane URL stays

48:13

stable. So I have it I have post plane,

48:15

which is another microservice I set up

48:16

that lets you upload a file with one

48:18

command, and as long as the path doesn't

48:20

change, the file uploads the same way

48:22

and will update with versions. And I

48:24

also say never open a browser or claim

48:25

the document's hosted before upload

48:26

succeeds. Do not verify in browser

48:27

unless the user asks. I was so tired of

48:29

Opus randomly doing that. This looks

48:31

much better. Let's go confirm the new

48:33

scale that we got here. Post plane read,

48:37

which shouldn't really need anything

48:38

special there, whatever. Fetch the

48:40

uploaded HTML with the shell. Do not use

48:42

web search or a browser. Remove trailing

48:44

{slash} then append {slash} raw unless

48:45

the URL already ends in {slash} raw. It

48:47

shouldn't actually need that. It should

48:48

be the same content now, but sure. Both

48:50

will work. Commit and push. Apply

48:52

changes to the fleet. In the real world,

48:56

I probably would have tidied this up

48:57

slightly more or went and confirmed that

48:59

the {slash} raw isn't necessary anymore

49:01

just to make these even simpler. Less

49:03

context is best as long as you has the

49:05

context it needs, but these are all

49:08

relatively simple, and hopefully this is

49:11

helpful. Use when the user asks to

49:13

communicate through an HTML document.

49:14

That's so much better. I kind of want to

49:15

add one more slight change here. Or if

49:18

they mention HTML with no additional

49:22

context. Cool. I want to be able to just

49:24

say HTML at the end of a prompt, and now

49:26

I have that. Cool. There. Just by adding

49:28

to the end of the description, if they

49:29

mention HTML with no additional context,

49:31

that means use this skill, which is what

49:33

I wanted. Sorry, made one tiny change to

49:36

description. Continue. And now those

49:39

updates will sync across my whole

49:40

machine in not very much time. This

49:43

shit's so cool. Like how cool is it that

49:46

I am writing markdown? I'm writing

49:48

effectively copy in order to change the

49:51

behavior of the the people, the agents

49:54

across all of the machines on my

49:55

network. And now this just works. It's

49:59

so cool. It's so cool. I could just tell

50:01

my agent to go sync all of this [ __ ] and

50:03

it figures it out just cuz I have SSH

50:05

access over tailscale setup. Igor

50:07

pointed out that Steve Jobs would be so

50:08

[ __ ] memorized by this. You just

50:10

Steve Jobs would go so hard at vibe

50:12

coding. He would have fired all of his

50:13

engineers and like turned macOS into

50:16

complete slop. You might have noticed

50:18

one thing I didn't do at any point in

50:19

this video. I never told you to go to

50:21

this repo and copy and install all of my

50:24

skills.

50:25

That's cuz I don't think that's a good

50:26

idea. Similar to how I don't think you

50:28

should just go install all of the

50:30

JavaScript packages somebody uses, you

50:32

shouldn't just go install all of the

50:34

skills someone uses. This is a thing

50:37

that you should build up based on your

50:38

needs and your experience communicating

50:41

with your agents on your projects. And

50:43

this is a great opportunity to level up

50:45

your own understanding of things as

50:46

well. In order to make good skills and

50:48

agent.md and Claude.md files, you need

50:50

to understand how your agents work and

50:51

where they fail. And hopefully this has

50:54

helped you see where I find these things

50:56

and you get some lessons you can apply

50:57

in your own work as well.

50:59

Don't blindly copy all the stuff I did

51:01

here. Take the opportunity to learn and

51:03

better understand how your agents work

51:04

in your stuff. And I have a feeling the

51:06

tips here will actually end up being

51:07

really helpful. This video is already

51:09

longer than Codex's system prompts. I'm

51:10

going to wrap it up now. I hope this was

51:12

helpful and until next time,

51:13

peace nerds.

Interactive Summary

Loading summary...