Hi, my name is Lauren. You might know me

as Potato OnX. And I work on Grockbot at

SpaceX AI. So, last month I did

something pretty crazy.

I shipped 2,000 pull requests to

production.

A lot of how I'm able to do this is

through trust. Um, and I think a lot

about trust in terms of how I can trust

my agents to produce high quality work

even when I'm not there. And my argument

and thesis for this talk today is that

if you set up your environment for your

agents really really well, you can end

up with something that looks more like a

personal or even team software factory

where you're producing very high quality

code at much greater rates than before.

Uh but I'm not a fan of the term

software factory. I like the analogy of

a Michelin kitchen better where you know

um as technologists we're not really

producing we're not mass producing a

product on an assembly line but the work

that we do looks very creative. It's

very um you know it's the act of

building a product and it's art in some

sense. So, you know, even though with

agents, we're not cooking the indivi

individual components that go into the

product anymore,

um, we're still responsible for the

final outcome and thinking about our

kitchen setup, right? Because depending

on how you set up your line cooks, your

sue chefs, you know, the kind of

equipment they have, the kind of

training they have, you know,

dishwashers and the ratio, I guess, of

line cooks to dishwashers. Um, all of

these ingredients go into making the

final product. Um, and I think this

analogy is really apt. So,

uh, I wanted to talk a little bit about

a story before we begin where six months

ago when I first joined Cursor before we

were a part of SpaceX AI, I obviously

had, you know, no agent skills to to

use, right? I just joined the company.

It was um a fresh code base and a fresh

product that I was working on.

So at the time um cursor was building

the replacement for the cursor IDE which

is the new agents window and uh before I

joined the cursor agent window had quite

a lot of performance issues. Um and my

manager at the time asked if I was able

to help them. Um and since I had spent

some time on the React team before I

joined Cursor uh it seemed like a good

fit. Uh but when I first started I

quickly realized how manual this process

was. Uh obviously I have done

performance work before. Um but at the

rate at which pull requests were being

landed it was just it felt almost an

insurmountable wall of pull requests

that just kept coming in and I had no

idea whether or not the performance of

the app would be regressing. Right? So a

lot of my early time on the cursor team

was spent looking at uh Chrome dev tools

and the performance and doing

performance traces and taking heap

snapshots.

Uh and it was extremely extremely

manual. It was it it was so manual it it

got to the point where um I just got

really frustrated and started to think

about you know wait we have agents what

am I doing? And so I started thinking

about verification skills where you know

what if my agent could actually run the

application itself and take the traces

for me, understand the traces and find

the hot spots and basically hill climb

you know to uh better uh performance in

our application automatically.

And throughout the last six months uh of

being at Cursor and SpaceX AI, you can

really see that my product PD has

skyrocketed.

And you know, I never set out to ship

2,000 pull requests a month. That was

not a goal of mine at all. But I

realized that all of the skills, all of

the tools and and codebased changes I

were making laded up to this idea of

trust. you know, I didn't know it at the

time, but um I had this thought in the

back of my head head, which was, you

know, I am the bottleneck and I need to

be able to take all of the knowledge

that I have as an engineer and impart

them into into my team of agents so that

I didn't need to uh be the blocker for

everything. And um you can clearly see

that it's it's paid off.

So

I think it really comes down to trust.

Um but how exactly do you build that

trust and where do you start if you are

you know wherever you are in your

journey of using agents. So uh when I

started obviously I was in this category

you know in the one to one to five range

where um you still feel like you have to

babysit every chat uh and end

conversation and you're just constantly

course correcting you know you're

intervening you're correcting your agent

so that it does the right thing. Um, and

if you're not there, basically nothing

gets nothing happens and the agents do

the wrong thing. And I would actually

argue that this part, this phase of, you

know, using agents is actually the

hardest to get out of because it's not

always very clear how exactly you get

out of it. And, um, again, it just comes

down to trust. um you know the reason

you're unable to go from one one or one

to five agents to something more like a

100 um is because you don't have trust

in your agents work yet. So if you don't

have trust and you try to spawn a 100

sub agents or cloud agents, you're going

to quickly find that you're just going

to get a ton of slot pull requests and

you know a bunch of regressions and a

bunch of bugs shipped and no one's going

to be very happy with that.

So that begs the question,

how do you trust your agents more?

So for me it came uh it it really

started when I uh like I said I meant I

joined the cursor team and I was

starting to work on performance where I

realized the need for verification.

So when I say verification uh there are

sort of levels to that uh because um you

know on the I guess the lower end of the

scale for verification you have things

like um the verification skills that I

talk about where you teach your agent

how to run your application um you know

and use things like the Chrome DevTools

protocol or whatever other protocol that

you have for debugging and teach them

how to

um you know debug the application take

performance traces

um take heat snapshots and so on. Um and

then on the opposite end of spectrum of

the spectrum which is much much harder

and still very much an open question is

like more formal verification where you

know maybe you rely on formal methods or

you know languages like lean or TLA plus

uh to do you know verification so that

you can check that you know your

business level or business logic uh

invariance are you

are always um true um and that um

uh you can formally verify that your

application is always in a correct

state. Uh but I would say that you know

even if you don't have um the ability to

run formal methods and very few people

really do um that with verification

skills you can get very far. So when I

joined cursor uh and started to work on

the cursor agent window, the first skill

that I built was the skill called

control glass which is a verification

skill that teaches the agent how to um

run the application and take traces like

I mentioned and it does that through uh

the chrome dev tool protocol.

Um something interesting about that

skill is uh and I kind of iterated my

way to this. Uh it didn't start out this

way. Uh but the control uh the control

or verification skill really has two

components to it. Um the first part is

obviously a CLI. So you want your agents

to be able to reproducibly

um be able to run the application and

collect traces and collect evidence that

you know empirical evidence that you

know code is working uh that your

performance bar is being met. Um, and

rather than have your agents create

scripts every time, um, you know, that

can differ between agent sessions, uh,

you can actually in you can actually

create a CLI that's within the skill

directory and then your agents will just

use that every time. And then of course

you need to actually invest in it and

make it good so that it can handle all

sorts of different use cases um and um

be able to uh you know run the

application uh correctly.

Uh another thing that's also really

important is uh this idea of a feature

map. So a feature map really is um

something that I kind of coined I guess

where uh you know we started using these

control skills within cursor. Then we

quickly realized that you know a Slack

report would come in and a user would

post a very vague screenshot right like

a very small slice of the UI and just

like three question marks and the agents

using our control skills had just no

idea like it could run the application

but it was just guessing at what exactly

the the user meant and so I had this

idea to create something called a

feature map um which is I guess kind of

inspired by like a site map Um it

essentially is a uh form of materialized

memory. You know like how exactly does

your application work? What features

does it have? How do does a user reach

it? Um you know in terms of keyboard

shortcuts or what DOM elements to click

on uh you know things like that and what

all the features do. Um and uh they are

these this feature map is stored in the

skill itself. um in the codebase as part

of the skill directory and we have an

automation that uh maintains this

feature map as well.

But when we combined the CLI and the

feature map, we quickly realized that um

this combination was very very powerful

because now agents could not only you

know reproducibly control the

application and pick traces but it could

also understand uh requests that came in

from internal users as well as external

users. And so we very quickly realized

that the control verification skills

were so useful that they've become more

or less uh critical infrastructure for

our team and we constantly uh maintain

it. But the ability for an agent to

verify its own work is extremely

powerful and uh we have um spent a lot

of time on this skill and it's very very

powerful for building that trust.

Um in addition to verification of course

uh you want uh because ver cor

verification is really about

correctness. Correctness to me is really

about does the thing does the feature or

code do the thing that you want it to

do, right? Like does the you know the

the checkout button does it actually

check out the card? Um verification is

really important for that because you

get empirical evidence that this feature

actually works but it doesn't tell you

much about you know the performance or

uh the code quality of that feature. Um,

and that's where you start thinking

about skills that teach agents to work

like real software engineers.

So,

um, I've built a plugin called PAC. Um,

I'm not going to talk about the plug-in

too much today, but um, a lot of the

inspiration for that plugin, uh, which

is a collection of skills that I've

created are inspired by the kind of

workflows that I personally have used as

my in my time doing software engineering

for all sorts of different types of

tasks. So debugging, you know, uh

feature development, prototyping,

there's a whole bunch of different

playbooks and skills that ship in PSA

that teach your agents how to write code

the way that you want them to. And you

know this is where uh you know the more

experienced engineers on your team can

really contribute to set up a team

repository of skills that um just make

your agents a lot smarter. Um, and when

you combine those skills with uh

verification skills, then you you're

able to get to a point where your agents

are able to not just verify that the

work that they're doing is correct, but

that it's also high quality. And again,

because of verification, you can collect

real performance metrics. you can get

real numbers and statistics and

telemetry on uh the like performance of

the application.

Um so I think that's a really important

part to invest in. Um

another thing I think is really

important is refactoring and rewriting

your architecture to be more agent

friendly. Um, and I almost want to say

that uh this is one of the most

important things you can do um as a as a

software engineering team because um if

you really truly believe that agents are

going to be writing all the code in the

future, then we need to design our code

bases so that they do the right thing by

default. And you'll find that there's

this uh I almost want to say like a a

scale or continuum between

uh you know these different pieces of

building trust in your agents where um

on you know I have like five of these

points here where uh

the code base is really like the best

form of memory because agents love to

extend existing patterns that they see.

Um, and you know, uh, I think this is

just the nature of how LLMs work where,

uh, they're more likely to use whatever

it is in their context window, um, to

make changes. And of course, the files

that the agents read and opens are part

of its context window. Um, and

therefore, the codebase is a really

important part of that. uh because you

know agents aren't going to just

refactor your code every single in every

single PR they're going to just look at

what's already there and just extend the

next level I think is about statical

analysis where you have llinters you

have compiler diagnostics you have uh

continuous integration

um and these are

guidelines and constraints that you can

enforce in your codebase so that

whenever you correct your agent, you

find that you know they just keep making

the same mistake.

Um you can add those as lint rules or

even better you can refactor your

codebase so that the mistake that the

agent is making becomes categorically

impossible.

Um and then a step above that right

where uh where and this is where we're

starting to get more into

uh less of like a hard constraint and

enforcement and more into the realm of

guidance. You have things like rules,

you have bugbot, you have skills um

which you know your agents will

obviously sometimes and mostly use uh

when they're doing their work, but

there's also a chance that it might for

various reasons you know forget to read

a rule or maybe the user that is

piloting the agent ignores them. Uh so

these aren't quite as enforceable but

also an important part of um

setting up your environment so that you

can really trust what your agents are

doing.

Uh and then finally you have the style

guide which is really only enforceable

by humans in a code review. I guess you

could put the put these in your rules

and and bug bot and skills as well. But

if you don't then you have this big

glaring hole in your uh review process

where now humans have to you know look

at every single line that's being

changed and remember to comment and with

the rate of pull requests that are

coming in it just it just becomes

impossible. So I definitely wouldn't

recommend you know relying only on the

style guide. I think the style guide or

you know like looking at human reviews

is a good place to start um in terms of

what's missing but you should really

invest the time to think about the other

four parts uh you know your codebase

making things categorically impossible

through better data structures or

algorithms

um static analysis and then of course

you layer that with rules and bug bot

and skills

Um on the codebase front um

and in the Grockbot codebase we

actually've invested into send setting

something up that we call Dune which is

our agentfriendly framework. So um the

inspiration for Dune really came about

from a lot of performance issues we're

seeing we were seeing in uh the cursor

agent window and so a lot of lessons

came out of that exploration but um the

key principle that we landed on is

really that agents love taking

shortcuts.

So what if we designed a framework such

that the shortcut you know the easy path

is the right path for agents and also

one that would be a codebase that is you

know maybe pretty annoying for humans to

work in uh because it's so locked down

in terms of what you can do and what you

can't do. But uh it actually creates the

perfect environment for agents

especially ones that have very minimal

context because you know not every

contributor to your codebase is going to

be an engineer anymore. You can have

designers, you can have product product

managers, you can have CEOs you know

going into the codebase and and shipping

features. So we want to really think a

lot about how we invest and set up our

code bases so that you know even agents

that are piloted by busy people with not

a lot of context can do a good job by

default.

And like I mentioned before um your code

base is really a form of memory for

agents because they love to extend the

existing patterns that they see. And the

reverse is actually also true, right?

You can invest the time to set up your

codebase um in a way that things are,

you know, bad patterns are categorically

impossible or you have like lit rules

that prevent them. But the reverse is

also true in the sense that um if you

have existing anti-atterns, you'll

actually find that these will spread

kind of like a virus. um where you have

like one small workaround or a comment

that explains a workaround

and you'll quickly find that agents just

love to copy that and then in a matter

of a few days or a few weeks you'll find

that the workound has spread everywhere

and it's now becomes it has become like

a de facto pattern for all agents and

that's a really really bad place to be

to be in and that goes back to what I

was saying about why it's really

important And you know whenever you're

correcting your your agents that you

invest the time into thinking about

codebased changes and static analysis to

and and of course the layering them with

rules good rules and bugbot and skills

uh because of of that reason.

Um, and another analogy that I like in

addition to the Michelin kitchen is this

idea that, you know, your code base is

kind of like a garden where you have uh

workarounds uh, you know, that seem

seemingly that seem kind of innocent at

first, but then because of the nature of

agency, you just copy that pattern over

and over again and you quickly end up

with a very, you know, vibecoded

codebase that is, you know, a pain in

the butt. to maintain and has a lot of

performance issues. So in my opinion,

the perfect agent codebase is one that's

so locked down that you know it's again

it's like really annoying for humans to

to write code in but it's so

conventional it's so standardized that

uh you know even innocent looking

patterns are just forbidden. Um, and the

best example of this I have is actually

something that seems very very innocent

when you look at it, but when you think

about it, it's actually really bad. And

that pattern is uh agents leaving

comments in the code. Now, you know,

when I first saw agents starting to do

this, I initially wasn't well, I I did

think that a lot of them were slob, but

I also thought that, you know, it

actually doesn't it's it's not a bad

thing, right? I guess if agents are

leaving comments in the code because as

humans we left comments in the code

whenever we saw you know edge cases or

we needed to actually make a workaround

or you know leave a note to ourselves or

a colleague on a particularly tricky

part of the codebase.

But what I quickly realized when we saw

this happening in the cursor codebase

was that agents were just using the

comments uh around the code as

justification for why it wasn't going to

solve the actual problem and instead

paper over it with a band-aid or a

short-term solution.

So in Dune, which is again the framework

that powers Grothbot, we made the choice

to um we made the choice to actually ban

comments for that reason where so that

agents would not just copy that pattern

and uh you know propagate it everywhere

in the codebase.

So uh my pitch here is that every team

really needs something that you know

role that I'm calling a gardener. Um in

the same way that uh you know with a

real garden you need someone who is

thinking a lot about the uh you know

things that can kind of creep in and

grow in ways that you don't want. like

you know you have weeds, you have uh you

know just other types of organic growth.

I don't really know gardening that well.

Um but you have things that you know

unwanted pests and and stuff like that

that kind of creep in in pure codebased

and so you want to nip them in the bud

as soon as possible before they start

propagating everywhere.

Um a lot of the principles behind Dune

are really centered around these three

things.

First of all, we want to delete tech

depth that we already have for, you

know, for reasons I just mentioned. We

want to keep uh or enforce a single pave

path for most blessed patterns.

uh you know there should be one

conventional way to do some things uh so

that agents don't really need to guess

and there should be enough guidance in

the codebase in CI

in lint rules uh so that the agents are

guided to do to to follow that path

and finally whenever you see tech or bad

patterns your instinct should be I need

to write a lint rule against it you

don't always have to clean it up

immediately uh because if you write a

lint rule, you can at least stop the

bleeding uh and which you know doesn't

solve the problem entirely but it at

least prevents it from growing. Um, so I

definitely recommend, you know, really

thinking a lot about how you can guard

against anti-atterns so that they don't

uh spread like a virus. And then also

spend time to actually, you know, get

your agents to clean them up so that

your codebase is just constantly kept in

a state where you would be happy if an

agent were to copy it. That's the kind

of mindset that I would recommend

having.

Uh and then I won't actually go through

all the details of Dune itself. Uh but

I'll just kind of gloss through some

interesting parts. Um so again as a

reminder Dune is the architecture the

client framework that we built to power

grabbot.

We've invested a lot into you know all

the things I was saying where we have

conventions. We have a lot of

conventions about where code should live

and um where and how uh code should be

imported between them. So in Dune

applications you know there's different

concepts where like for example features

are all coll-located in a single folder

um you have an entry point um that's you

know in the react part of the code that

determines or you can kind of think of

it like a route uh you have transcript

cards that show up in the graphbot

application you have a host that runs on

the you know the graphbot virtual

machine and then of course you have your

client uh which uh powers as far as the

uh overall gene application. And we have

a lot of um strict boundaries between

these things where um just an as an

example, things that run on the main

process uh or the main thread in

Electron aren't allowed to be run on the

renderer thread. And we keep that

separation very intentionally because uh

of lessons we learned from cursors agent

window where we would sometimes see code

accidentally uh get imported into the

renderer thread and you know slow code

and uh since uh on the renderer thread

you want your UI to be very smooth and

uh performant uh you need to make sure

that you don't have any long tasks or

you know things that take longer than 16

milliseconds for if you want like 60

frames per second or 8 milliseconds if

you want 120 frames per second. And so

your renderer has to be constantly in a

state where uh it is uh it can really

kind of chunk up the work um and not do

them all at once.

Um and so we have code within Dune that

enforces this these boundaries through

the import uh and dependency graph. Uh

but yeah, that's just an example of a

pattern that we saw lead to really bad

performance that we um categorically

eliminated through uh the architecture

of the of Dune.

Um and then all of these other pieces

aren't that interesting. Uh but again

the the core theme here you know it's

not about Dune but the idea that um an

agent friendly framework of your own is

actually very very powerful and you can

encode all of the learnings that you and

your you know your best engineers on

your team have tribal knowledge of. Um

and um I think the lesson here is that

how do you take that away from you know

what used to be in the style guide

process of reviewing code and you know

engineers reviewing other engineers work

and leaving comments

um to extract almost like extracting

that knowledge and encoding that into

the framework into the codebase itself

so that the codebase access the memory

right it's the the thing I keep coming

back to this idea that you know the

codebase is just the thing that it's the

the materialized snapshot of the state

in which you want your agents to extend

and you want that code base to be so

pristine so great that the next agent

that comes along is just very likely to

continue that pattern and keep it really

really good.

Um, and if you spend enough time on this

process, uh, like I mentioned, you can

really set up, uh, a Michelin kitchen or

a software factory where,

um, because you spent so much time on,

you know, all of these pieces that allow

you to trust your agents, whether it's

in the codebase, whether it's lint

rules, um, whether it is,

uh, you know, diagnostics or rules or

bug bot or skills. These layers come

together um and provide you a lot of

trust because now you know just imagine

for a moment you're working in Rockbot

codebase. Um

it's super locked down. You know it's

like almost impossible to write bad

code. So you know you can even an agent

with very little context. uh you know

even a agent with not a lot of reasoning

can come in and and actually write code

that's good and going back to my example

about the Michelin factory I think

there's a lot here right where you know

we're setting up our agents our bots

with skills and tools you know we're

training them uh we're setting up our

kitchen in a way that makes sense right

for the agents and bots to do the right

thing by default uh uh you know whenever

we see for example in in a kitchen

example if we notice that uh one of our

cooks or dishwashers is constantly

tripping over something of course we

need to fix that right we need to

problem solve and ensure that you know

others don't trip as well because you

know in a kitchen it's a very dangerous

place and you don't want to hurt

yourself it's the same

mindset I think that we should have with

our code bases how do we set it up so

that even agents uh without a lot of uh

knowledge can and do a good job. Um, and

I think with uh, you know, Grockbot,

Grockbot and cursor play an interesting

role together where Grockbot is really

great at um, providing what I call the

outer loop because you can connect

Grockbot to lots of different uh, you

know, different connectors like Slack to

Data Dog, Sentry,

uh, Planet Scale, whatever services that

you use and you can aggregate all of

that information together and use that

to

really good decisions for itself. Uh

some people call this like a company

brain. Uh I don't really think I

personally don't think you need anything

that sophisticated here because agents

are really good at using tools. And so

if you connect these tools to Grockbot

um and you start having your Grock bots

auto kick off things like cloud agents,

you can actually find that it's really

not uh you don't really have to invest

in a lot of infrastructure to build a

software factory. Um in fact, I'm gonna,

you know, cross cross out this this term

because I don't like this term. Um uh I

think you can set up this personal

Michelin Kitchen for yourself through

Grobot things like Grockbot routines

which let you subscribe to you know

Slack threads to sentry alerts that let

you kick off things automatically.

And when you combine all of these things

that I've been mentioning, you know,

your codebase, your rules, your skills,

uh they all compound and Grockbot uh

will be able to, you know, automatically

respond to events that come from the

outer loop and then kick off cloud

agents. Um, and you can also set up

cursor automations and use our SDK uh to

set up um bot additional bots as well

that reuse a lot of these pieces of

agent infra that you've set up um and

allow them to do much more complicated

tasks.

So if you if you if you've done all this

then I think you can get to a point

where uh you know uh I have some

screenshots here of some of our

automations and our agents in uh that

that work on cursor where we are

automatically

reproducing bug reports. We're

automatically opening pull requests. we

are um essentially adding a lot of value

to the entire team uh because all of

these things compound.

So if we kind of zoom out again um and

go back to this graph,

I think that to kind of close off the

talk, um if you spend a lot of time

thinking about all of the pieces that

you need uh to be able to ascend the

trust graph, uh you start getting to a

place where you can uh really trust your

agents more and parallelize your work

and also empower your entire team to

build on top of these uh pieces of

infrastructure for your agents um and

empower everyone, you know, every

engineer on your team, every builder to

be extremely productive and be able to

write high quality code. So, the last

thing I want to leave you with is

actually this piece.

Uh sorry, not that piece, but this

piece. Um I think if there's only one

thing you take away from my talk, it

should be this this slide here. uh which

is you know these are the activities

that will help you build up towards a

high trust environment. You know,

whenever you find yourself correcting

and intervening your agent, um you

really want to think about it from these

five pieces and where is the most

effective step in this um sequence uh in

order to uh get make your agent you know

much more trustworthy. Um and of course

I definitely recommend thinking about

thinking about it in this order where

you know you uh either invest the time

to make that pattern uh categorically

impossible through your codebase and

architecture and data structures or you

start looking at things like static

analysis and then you layer that on with

rules and bugbot and skills.

If you do all of that and you also spend

some time, you know, thinking about uh

your code quality uh in in terms of of

skills, you get to a place where you

trust the you trust the environment so

much that your agents can just be free,

right? And uh personally, I have spent a

lot of time for this for Grockbots's

codebase for example and uh this is

really the secret, right? Well, it's not

really a secret. It's it's a lot of hard

work. But um I hope you found this talk

useful and uh please reach out to me on

X. Uh my my handle is potato with an E.

Um and uh I hope that you'll have a lot

of fun and success pulling your own

Michelin Kitchen. Thanks for watching.