Hi, my name's Lauren. You might know me

as potato on X and I work on Grok Bot at

SpaceX AI.

So, last month I did something pretty

crazy.

I shipped 2,000 pull requests to

production.

A lot of how I'm able to do this is

through trust.

Um and I think a lot about trust in

terms of

how I can trust my agents to produce

high-quality work even when I'm not

there.

And my argument and thesis for this talk

today is that if you set up your

environment for your agents really,

really well,

you can end up with something that looks

more like a personal or even team

software factory

where you're producing very high-quality

code at much greater rates than before.

Uh but I'm not a fan of the term

software factory.

I like the analogy of a Michelin kitchen

better

where, you know, um as technologists,

we're not really producing we're not

mass producing a product on an assembly

line.

But the work that we do looks very

creative. It's very

um you know, it's the act of building a

product. And it's art in some sense.

So, you know, even though with agents

we're not

cooking the individual components that

go into the product anymore,

um

we're still responsible for the final

outcome

and thinking about our kitchen setup.

Right? Because depending on how you set

up your line cooks, your sous chefs, you

know, the kind of equipment they have,

the kind of training they have,

you know, dishwashers and the ratio I

guess of line cooks to dishwashers,

um all of these ingredients go into

making the final product. Um and I think

this analogy is really apt.

So,

uh I wanted to talk a little bit about a

story before we begin.

Where 6 months ago, when I first joined

Cursor, before we were a part of SpaceX

AI,

I obviously had, you know, no agent

skills to to use, right? I just joined

the company. It was

um a fresh code base and a fresh product

that I was working on.

So, at the time,

um Cursor was building the replacement

for the Cursor IDE,

which is the new agents window.

And uh before I joined, the Cursor agent

window had quite a lot of performance

issues.

Um and my manager at the time asked if I

was able to help them.

Um and since I had spent some time on

the React team before I joined Cursor,

uh it seemed like a good fit.

Uh but when I first started, I quickly

realized how manual this process was.

Uh obviously I've done performance work

before,

um but at the rate at which pull

requests were being landed, it was just

it felt almost an insurmountable wall of

pull requests that just kept coming in

and I had no idea whether or not the

performance of the app would be

regressing,

right?

So, a lot of my early time on the Cursor

team was spent looking at uh Chrome

DevTools and the performance and doing

performance traces and taking heap

snapshots.

Uh and it was extremely extremely

manual.

It was it It was so manual it it got to

the point where um I just got really

frustrated and started to think about,

you know, wait, we have agents, what am

I doing?

And so, I started thinking about

verification skills. Where you know,

what if my agent could actually run the

application itself and take the traces

for me, understand the traces, and find

the hot spots, and basically hill climb,

you know, to uh better

uh performance in our application

automatically.

And throughout the last 6 months uh of

being at Cursor and SpaceX AI,

you can really see that my productivity

has skyrocketed.

And, you know, I never set out to ship

2,000 pull requests a month. That was

not a goal of mine at all.

But, I realized that all of the skills,

all of the tools, and and code base

changes I were making

laddered up to this idea of trust. You

know, I didn't know it at the time, but

um I had this

thought in the back of my head head,

which was, you know, I am the

bottleneck,

and I need to be able to take all of the

knowledge that I have as an engineer and

impart them into into my team of agents

so that I didn't need to

uh be the blocker for everything. And um

you can clearly see that it's it's paid

off.

So,

I think it really comes down to trust.

Um but, how exactly do you build that

trust? And where do you start if you

are, you know, wherever you are in your

journey of using agents?

So, uh when I started, obviously, I was

in this category, you know, the one to

one to five range,

where

um you still feel like you have to baby

sit every chat uh and and conversation,

and you're just constantly

course correcting, you know, you're

intervening, you're correcting your

agent so that it does the right thing.

Um and if you're not there, basically

nothing get nothing happens and the

agents do the wrong thing.

And I would actually argue that this

part this phase of, you know, using

agents is actually the hardest to get

out of

because it's not always very clear how

exactly you get out of it.

And um again, it just comes down to

trust.

Um you know, the reason you're unable to

go from one one or one to five agents to

something more like 100

um is because you don't have trust in

your agents work yet.

So, if you don't have trust and you try

to spawn 100 sub agents or cloud agents

you're going to quickly find that you're

just going to get a ton of slot pull

requests and you know, a bunch of

regressions and a bunch of bugs shipped

and no one's going to be very happy with

that.

So, that begs the question

how do you trust your agents more?

So, for me it came uh it it really

started when I uh like I said, I meant I

joined the cursor team and I was

starting to work on performance where I

realized the need for verification.

So, when I say verification uh there are

sort of levels to that.

Uh because um you know, on the I guess

the lower end of the scale for

verification, you have things like um

the verification skills that I talk

about where you teach your agent how to

run your application

um you know, and use things like the

Chrome DevTools protocol or whatever

other protocol that you have for

debugging and teach them how to um

you know, debug the application, take

performance traces um

take heap snapshots and so on.

Um and then on the opposite end of

spectrum of the spectrum, which is much

much harder and still very much an open

question,

is like more

formal verification, where, you know,

maybe you rely on uh formal methods or,

you know, languages like Lean or TLA+

uh to do, you know, verification so that

you can check that, you know, your

business level or business logic uh

invariants are, you know, are always um

true.

Um and that um

uh you can formally verify that your

application is always in a correct

state.

Uh but I would say that, you know, even

if you don't have um the ability to run

formal methods, and very few people

really do,

um that with verification skills, you

can get very far.

So, when I joined Cursor, uh and started

to work on the Cursor agent window, the

first skill that I built was the skill

called control glass,

which is a verification skill that

teaches the agent how to

um run the application and take traces,

like I mentioned,

and it does that through

uh the Chrome DevTools protocol.

Um something interesting about that

skill

is uh and I kind of iterated my way to

this, uh it didn't start out this way,

um but the control

uh the control or verification skill

really has two components to it.

Um

the first part is obviously a CLI.

So, you want your agent to be able to

reproducibly

um be able to run the application and

collect traces and collect evidence

that, you know, empirical evidence that,

you know, code is working,

uh that your performance bar is being

met. Um and rather than have your agents

create scripts every time,

um you know, that can differ between

agent sessions, uh you can actually in

you can actually create a CLI that's

within the skill directory, and then

your agents will just use that every

time.

And then,

of course, you need to actually invest

in it and make it good so that it can

handle all sorts of different use cases,

um

and um

be able to

uh you know, run the application

correctly.

Uh another thing that's also really

important is uh this idea of a feature

map.

So, a feature map really is um something

that I kind of coined, I guess, where uh

you know, we started using these control

skills with Incursor.

Then we quickly realized that, you know,

a Slack report would come in and a user

would post a very vague screenshot,

right? Like a very small slice of the

UI, and just like three question marks.

And the agents using our control skills

had just no idea. Like it could run the

application, but it would just be

guessing at what exactly the the user

meant.

And so, I had this idea to create

something called a feature map,

um which is

I guess kind of inspired by like a site

map.

Um it essentially is a

uh form of materialized memory, you

know, like how exactly does your

application work? What features does it

have? How do it does a user reach it? Um

you know, in terms of like keyboard

shortcuts or what DOM elements to click

on, uh you know, things like that, and

what all the different features do.

Um and uh they are these this feature

map is stored in the skill itself,

um in the code base as part of the skill

directory.

And we have an automation that uh

maintains this feature map as well.

But when we combine the CLI and the

feature map, we quickly realized that um

this combination was very, very powerful

because now agents could not only,

you know, reproducibly control the

application and take traces,

but it could also understand

uh requests that came in from internal

users as well as external users.

And so, we very quickly realized that

the control verification skills

were so useful that they've become more

or less

uh critical infrastructure for our team,

and we constantly uh maintain it.

But the ability for an agent to verify

its own work is extremely powerful,

and uh we have um

spent a lot of the time on the skill,

and it's very, very powerful for

building that trust.

Um in addition to verification, of

course,

uh you want uh because ver- uh

verification is really about

correctness.

Correctness to me is really about does

the thing does the feature or the code

do the thing that you want it to do?

Right? Like does the you know, the the

checkout button, does it actually check

out the cart?

Um verification's really important for

that because you can get empirical

evidence that the feature actually

works.

But it doesn't tell you much about, you

know, the performance or

uh the code quality of that feature.

Um and that's where you start thinking

about skills that teach agents to work

like real software engineers.

So,

um I've built a plugin called Pystack.

Um I'm not going to talk about the

plugin too much today.

But

um a lot of the inspiration for that

plug-in,

uh which is a collection of skills that

I've created,

are inspired by the kind of workflows

that I personally have used as my in my

time doing software engineering

for all sorts of different types of

tasks. So, debugging, you know,

uh feature development, prototyping.

There's a whole bunch of different

playbooks and skills that ship in Piece

That that teach your agents how to write

code the way that you want them to.

And you know, this is where

uh you know, the more experienced

engineers on your team can really

contribute

to set up a team repository of skills

that um just make your agents a lot

smarter.

Um

and when you combine those skills with

uh verification skills,

then you you're able to get to a point

where your agents are able to not just

verify that the work that they're doing

is correct, but that it's also high

quality.

And again, because of verification, you

can collect real performance metrics.

You can get real numbers and statistics

and telemetry

on uh the like performance of the

application.

Um so, I think that's a really important

part to invest in.

Um

Another thing I think is really

important is refactoring and rewriting

your architecture to be more

agent-friendly.

Um and I almost want to say that

uh this is one of the most important

things you can do

um as a as a software engineering team

because um if you really truly believe

that agents are going to be writing all

the code in the future,

then we need to design our code bases so

that they do the right thing by default.

>> Mhm.

And you'll find that there's this

uh I I almost want to say like a

a scale or continuum between

uh you know, these different pieces of

building trust in your agents

where

um on you know, I have like five of

these points here where

uh

the code base is really like the best

form of memory

because agents love to extend existing

patterns that they see.

Um and you know, uh I think this is just

the nature of how LLMs work where uh

they're more likely to use whatever it

is in their context window um to make

changes. And of course, the files that

the agents read and opened are part of

its context window.

Um and therefore, the code base is a

really important part of that.

Um because you know, agents aren't going

to just refactor your code every single

in every single PR. They're going to

just look at what's already there and

just extend. The next level, I think, is

about static analysis

where you have linters, you have

compiler diagnostics, you have

uh continuous integration

um

and these are

guidelines and constraints that you can

enforce in your code base so that

whenever you correct your agent, you

find that, you know, they just keep

making the same mistake.

Um you can add those as lint rules or

even better, you can refactor your code

base so that the mistake that the

agent's making becomes categorically

impossible.

Um and then a step above that, right?

Where uh where and this is where we're

starting to get more into

uh less of like a hard constraint and

enforcement and more into the realm of

guidance.

You have things like rules, you have bug

bot, you have skills,

um which, you know, your agents will

obviously sometimes and mostly use uh

when they're doing their work, but

there's also a chance that it might, for

various reasons, you know, forget to

read a rule or maybe the user that is

piloting the agent ignores them.

Uh so, these aren't quite as

enforceable, but also an important part

of

um

setting up your environment so that you

can really trust what your agents are

doing.

Uh and then finally, you have a style

guide, which is really only enforceable

by humans in a code review. I guess you

could put the put these in your rules

and and bug bot and skills as well.

But if you don't, then you have this big

glaring hole in your

uh review process, where now humans have

to, you know, look at every single line

that's being changed and remember the

comment and

with the rate of pull request that are

coming in, it just it just becomes

impossible.

So, I definitely wouldn't recommend, you

know, relying only on the style guide. I

think the style guide or, you know, like

looking at human reviews is a good place

to start

um in terms of what's missing,

but you should really invest the time to

think about the other four parts,

uh you know, your code base, making

things categorically impossible to

better

data structures or algorithms,

um static analysis, and then of course,

you layer that with rules and bug bot

and skills.

Um on the code base front,

um

and in the Grokbot code base,

we actually have invested into

setting something up that we call

Dune, which is our agent-friendly

framework.

So,

um the inspiration for Dune really came

about from a lot of the performance

issues we were seeing we were seeing in

um the cursor agent window.

And so, a lot of lessons came out of

that exploration,

but um

the key principle that we landed on is

really that agents love taking

shortcuts.

So, what if we designed a framework such

that the shortcut, you know, the easy

path is the right path for agents. And

also one that would be a code base that

is, you know, maybe pretty annoying for

humans to work in uh because it's so

locked down in terms of what you can do

and what you can't do,

but uh it actually creates the perfect

environment for agents, especially ones

that have very minimal context because,

you know, not every contributor to your

code base is going to be an engineer

anymore. You can have designers, you can

have product product managers, you can

have CEOs, you know, going into the code

base and and shipping features. So, we

want to really think a lot about how we

invest and set up our code bases so

that, you know, even agents that are

piloted by busy people with not a lot of

context can do a good job by default.

And like I mentioned before,

um

your code base is really a form of

memory for agents because they love to

extend the existing patterns that they

see.

And the reverse is actually also true,

right? You can invest the time to set up

your code base um in a way that things

are, you know, bad patterns are

categorically impossible, or you have

like lint rules that prevent them,

but the reverse is also true in the

sense that um if you have existing

anti-patterns, you'll actually find that

these will spread kind of like a virus.

Um

where you have like one small workaround

or a comment that explains a workaround,

and you'll quickly find that agents just

love to copy that. And then in a matter

of a few days or a few weeks, you'll

find that the workaround has spread

everywhere and it's now becomes it has

become like a de facto pattern for all

agents. And that's a really, really bad

place to be to be in.

And that goes back to what I was saying

about why it's really important, you

know, whenever you're correcting your

your agents,

that you invest the time into thinking

about codebase changes and static

analysis to and and of course the

layering them with rules, good rules,

and bug bot and skills

uh because of of that reason.

Um and

another analogy that I like in addition

to the Michelin kitchen is this idea

that, you know, your codebase is kind of

like a garden

where you have

uh

workarounds uh you know, that seem

seemingly that seem kind of innocent at

first, but then because of the nature of

agents, you just copy that pattern over

and over again,

and you quickly end up with a very, you

know, vibe coded codebase that is a you

know, a pain in the butt to maintain and

has a lot of performance issues.

So, in my opinion, the perfect agent

codebase is one that's so locked down

that, you know, it's again it's like

really annoying for humans to to write

code in,

but it's so conventional, it's so

standardized that

uh you know, even innocent-looking

patterns are just forbidden.

Um and the best example of this I have

is actually something that seems very,

very innocent when you look at it,

but when you think about it, it's

actually really bad.

And that pattern is

uh agents leaving comments in the code.

Now, you know, when I first saw agents

starting to do this, I

initially wasn't Well, I I did think

that a lot of them were slop, but I also

thought that, you know, it actually

doesn't It's It's not a bad thing,

right? I guess if agents are leaving

comments in the code because

as humans, we left

comments in the code whenever we saw,

you know, edge cases or we needed to

actually make a workaround or, you know,

leave a note to ourselves or a colleague

on a particularly tricky part of the

code base.

But,

what I quickly realized when we saw this

happening in the Cursor code base

was that agents were just using the

comments

around the code as justification for why

it wasn't going to solve the actual

problem.

And instead paper over it with a

band-aid or

a short-term solution.

So, in Dune, which is again the

framework that powers Grok Bot,

we made the choice to

um

we made the choice to actually ban

comments

before that reason where so that agents

would not just copy

that pattern and uh

you know, propagate it everywhere in the

code base.

So, uh

my pitch here is that every team really

needs something that, you know, like a

role that I'm calling a gardener.

Um in the same way that uh you know,

with a real garden, you need someone who

is thinking a lot about

the

uh you know, things that can kind of

creep in and grow

in ways that you don't want. Like, you

know, you have weeds, you have

uh you know, just other types of organic

growth.

I don't really know much about gardening

that well. Uh I

But you have things that you know

unwanted pests and and and stuff like

that that kind of creep in into your

code base.

And so you want to nip them in the bud

as soon as possible before they start

propagating everywhere.

A lot of the principles behind Dune are

really centered around these three

things.

First of all, we want to delete tech

debt that we already have for you know

for reasons I just mentioned.

We want to

keep or enforce a single paved path for

most blessed patterns.

You know there should be one

conventional way to do some things.

So that agents don't really need to

guess and there should be enough

guidance in the code base in CI

in lint rules

so that the agents are guided to do to

to follow that path.

And then finally whenever you see tech

debt or bad patterns, your instinct

should be I need to write a lint rule

against it.

You don't always have to clean up

immediately

because if you write a lint rule, you

can at least stop the bleeding.

And which you know doesn't solve the

problem entirely, but it at least

prevents it from growing.

So I definitely recommend you know

really thinking a lot about

how you can guard against anti-patterns

so that they don't spread like a virus

and then also spend time to actually you

know get your agents to clean them up so

that your code base is just constantly

kept in a state where

you would be happy if an agent would

have copied it. That's the kind of

mindset that I would recommend having.

And then I won't actually go through all

the details of Dune itself. But I'll

just kind of gloss through some

interesting parts.

So again as a reminder, Dune this

architecture the client framework that

we built to power GraphBot.

We've invested a lot into, you know, all

the things I was saying, where we have

conventions. We have a lot of

conventions about where code should

live.

And

um where and how

uh code should be imported between them.

So, in Dune applications, you know,

there's different concepts where like,

for example, features are all co-located

in a single folder.

Um you have an entry point um that's,

you know, in the React part of the code

that determines uh you can kind of think

of it like a route.

Uh you have transcript cards that show

up in the GraphBot application.

You have a host that runs on the

you know, the GraphBot virtual machine,

and then of course, you have your client

uh which uh powers the uh overall Dune

application.

And we have a lot of um strict

boundaries between these things, where

um

just an as an example, things that run

on the main process

uh or the main thread in Electron aren't

allowed to be run on the renderer

thread. And we keep that separation very

intentionally because uh of lessons we

learned from Cursor's agent window,

where we would sometimes see

code accidentally uh get imported into

the renderer thread, and you know, slow

code. And uh since

uh on the renderer thread, you want your

UI to be very smooth and

uh performant, uh you need to make sure

that you don't have any long tasks or,

you know, things that take longer than

16 milliseconds for if you want like 60

frames per second, or 8 milliseconds if

you want 120 frames per second.

And so, your renderer has to be

constantly in a state where

uh it is uh you can really kind of chunk

up the work um, and not do them all at

once.

Um, and so, we have code within Dune

that enforces this

these boundaries through the import uh,

independency graph.

Uh, but, yeah, this is just an example

of a pattern that we saw lead to really

bad performance that we um,

categorically eliminated through uh,

the architecture of the of Dune.

Um,

and then, all of these other pieces

aren't that interesting.

Uh,

but, again, the

the core theme here you know, it's not

about Dune but the idea that

um,

an agent-friendly framework of your own

is actually very, very powerful.

And you can encode all of the learnings

that you and your, you know, your best

engineers on your team have

tribal knowledge of. Um, and um, I think

the lesson here is that how do you take

that away

from, you know, what used to be in the

style guide process of reviewing code

and, you know, in engineering is

reviewing other engineers' work and

leaving comments

um, to

extract, almost like extracting that

knowledge and encoding that into the

framework

into the code base itself so that the

code base acts as the memory. Right?

It's the the thing, like you're coming

back to this idea that, you know, the

code base is just the thing that it's

the

the materialized snapshot of the state

in which you want your agents to extend.

And you want that code base to be so

pristine, so great that the next agent

that comes along is just very likely to

continue that pattern and keep it

really, really good.

Um, and if you spend enough time on this

process

uh, like I mentioned, you can really set

up um, a Michelin kitchen or a software

factory

where

um, because you spent so much time on

you know, all of these pieces that allow

you to trust your agent,

whether it's in the code base, whether

it's lint rules,

um, whether it is

uh, you know, diagnostics or rules or

bug bar or skills,

these layers come together

um, and provide you a lot of trust.

Because now, you know, just imagine for

a moment, you're working in the GrokBot

code base.

Um,

it's super locked down, you know, it's

like almost impossible to write bad

code. So, you know, you can even an

agent with very little context,

uh, you know, even a an agent with not a

lot of reasoning can come in and and

actually write code that's good.

And going back to my example about the

Michelin factory, I think there's

a lot here, right? Where, you know,

we're setting up our agent, our bot with

skills and tools, you know, we're

training them, uh, we're setting up our

kitchen in a way that makes sense,

right? For

the agents and bots to do the right

thing by default.

Uh, you know, whenever we see, for

example, in the in the kitchen example,

if we notice that uh, one of our cooks

or dishwashers is constantly tripping

over something, of course we need to fix

that. Right? We need to problem solve

and ensure that, you know, others don't

trip as well because, you know, in a

kitchen is a very dangerous place and

you don't want to hurt yourself.

It's the same

mindset, I think, that we should have

with our code bases. How do we

set it up so that even agents, uh,

without a lot of

uh, knowledge can and do a good job.

Um, and I think with

uh, you know, GrokBot,

GrokBot and Cursor play an interesting

role together where GrokBot is really

great at uh providing what I call the

outer loop because you can connect

GrokBot to lots of different uh you

know, different connectors like Slack to

DataDog, Sentry,

uh PlanetScale, whatever services that

you use. And you can aggregate all of

that information together and use that

to make really good decisions for

itself.

Uh

Some people call this like a company

brain. Uh I don't really think I

personally don't think you need anything

that sophisticated here because agents

are really good at using tools. And so,

if you connect these tools to GrokBot

um and you start having your GrokBots

auto kick off things like cloud agents,

you can actually find that it's really

not uh

you don't really have to invest in a lot

of infrastructure to build a software

factory.

Um in fact, I'm going to you know, cross

cross out this

this term cuz I don't like this term.

Um uh I think you can set up this

personal mission control kitchen for

yourself through GrokBot, things like

GrokBot routines which let you subscribe

to you know, Slack threads to Sentry

alerts that let you kick off things

automatically.

And when you combine all of these things

that I've been mentioning, you know,

your code base, your rules, your skills,

uh they all compound and GrokBot uh will

be able to you know, automatically

respond to events that come from the

outer loop and then kick off cloud

agents. Um and you can also set up

Cursor automations and use our SDK uh to

set up um bot additional bots as well

that reuse a lot of these

pieces of agent infra that you've set up

um and allow them to do much more

complicated tasks.

So, if you if you if you've done all

this, then I think you can get to a

point where

uh you know, uh I have some screenshots

here of some of our automations and our

agents in

uh that that work on cursor

where we are automatically re-

reproducing bug reports, we're

automatically opening pull requests, we

are um essentially adding a lot of value

to the entire team

uh because all of these things compound.

So,

if we kind of zoom out again,

um and go back to this graph,

I think that to kind of close off the

talk,

um if you spend a lot of time thinking

about all of the pieces that you need uh

to be able to ascend the trust graph,

uh you start getting to a place where

you can

uh really trust your agents more and

parallelize your work, and also empower

your entire team to build on top of

these uh pieces of infrastructure for

your agents, um and empower everyone,

you know, every engineer on your team,

every builder,

to be extremely productive and be able

to write high-quality code.

So, the last thing I want to leave you

with is actually this piece.

Uh sorry, not that piece, but this

piece.

Um I think if there's only one thing you

take away from my talk, it should be

this this slide here,

uh which is, you know, that these are

the activities that will help you build

up towards a high-trust environment. You

know,

whenever you find yourself correcting

and intervening your agent,

um you really want to think about it

from these five pieces and where is the

most effective step in this

sequence

in order to

get make your agent you know much more

trustworthy

and of course I definitely recommend

thinking about thinking about it in this

order where you know you

either invest the time to make that

pattern categorically impossible through

your code base and architecture and data

structures or you start looking at

things like static analysis and then you

layer that on with rules and bug bot and

skills.

If you do all of that and you also spend

some time you know thinking about your

code quality

in in terms of of skills

you get to a place where you trust the

you trust the environment so much that

your agents can just be free

right and personally I have spent a lot

of time for this for growth bots code

base for example and

this is really the secret right well

it's not really a secret it's it's a lot

of hard work but I hope you found this

talk useful

and please reach out to me on X my

handle is potato with an e

and

I hope that you'll have

a lot of fun and successfully in your

own mission in kitchen.

Thanks for watching.