Everyone, I am Lauren Lauren Tan. I

guess not many people know my last name.

Uh, I am Potato on Twitter. Uh, potato

with spelled with an E. Um, and I have

been at Cursor for about 5 months. Uh,

previously I was at Meta where I worked

on the React team. Uh, specifically

working on the React compiler, uh, which

was a whole lot of fun. Uh I'm still on

the on the core team and and uh

contributing to open source here and

there. Uh so that that's really nice

that they still let me do that. Uh and

before Meta, I was at Netflix uh where I

was uh both a tech lead and uh I

transitioned to be be an engineering

manager um for about two years. So I've

had a I've have had a lot of experience

going between engineering management and

being an individual contributor. Uh and

I think something I've noticed actually

which is quite interesting is that there

are so many parallels with you know

management skills and how to like manage

agents. Uh and that's actually a big

part about what I wanted to chat with

you and everybody else about today. Um

but yeah that's that's me. Uh I do have

some like very light slides but uh it's

not going to be uh just rambling. So let

me just share my screen

and hope that I don't leak anything. Uh

oh no I need to allow permissions.

>> No worries. Take your time. There's

always tech tech trouble. Uh give me one

second to rejoin.

>> Yeah, go for it.

I see many of you already know Lauren

from from the looks of the chat here.

Um, so yeah, it's exciting to to get a

chance to chat with her uh and and go

through some of her her recent work. As

you guys heard, you know, a lot of

recent experience from from Netflix to

to Meta and then now over at Cursor. Uh,

we're going to chat a little bit about

Grockbot as well. So, uh, that'll be

exciting. I don't know if you guys saw

that was a recent release. I think

literally maybe yesterday or the day

before uh from the cursor team which is

kind of like um let's call it like

agents for everyone. You can go check it

out if you want and learn a little bit

more about the product but um but yeah

we'll we'll explore that a little bit

today as well.

All righty. Welcome back.

And Lauren, you're just on mute there if

uh you want to hop off of mute if you're

chatting.

>> Yeah, sorry.

>> No worries.

>> It's 2026 and I still don't know how to

use Zoom. [laughter]

>> That's all good.

>> Uh okay, so I assume you can see my

screen.

>> Yes. Yeah, we're good. Um, so yeah,

today, yeah, I think I think the big

theme for me as I've been using agents

to write code, and I'm sure a lot of you

have had the same experience as well, is

how do you trust it? You know,

especially if you are an engineer that's

been writing code for a very long time,

you have a lot of opinions and lessons

that you've learned about doing good

engineering. And when you see agents

just, you know, winging it and, you

know, guessing, hallucinating,

uh, you know, confidently stating that

they found the smoking gun, uh, for the

hundth time, uh, but it's actually not

the real problem. You lose a lot of

trust. And when you lose when you don't

have much trust in your agents,

I feel like you you really can't get the

most out of them. And for me, the

parallel is like with management. Uh so

if I'm an a manager an engineering

manager of a team and I have a bunch of

you know I have a team of engineers uh

on my team and I don't trust them then

the mode of operation I'm going to be in

is going to be like micromanagement

right I'll have to spend a lot of time

looking over my reports shoulders and

checking that they're doing their work

well you know that they're not shipping

bugs to production

and so I drew this chart cuz uh it's not

it's not a very scientific chart but

like this is how I imagine myself and my

journey through using agents. So you

know like fast forward or back forward

uh or fast back uh fast backwards like a

year or so when you know nobody was or

not many people were using agents to

code. Uh I think you

uh you know get into this mode where you

are

in very heavily in the loop with one or

several like a handful of agents and you

find yourself just constantly fig uh you

know trying to understand what your

agents are doing uh and you're very very

in the loop. You're watching every

single output. you are sitting there

prompting um and you really can't

parallelize beyond that because you

don't again you don't have that trust

right you can't go to a 100 agents uh

like spawn 100 agents when you don't

even trust the output of one agent

so over the past 5 months I feel like

I've really been able to uh like ascend

this trust curve and now I'm at the

point where uh I actually have this

Sounds kind of scary to say this and it

it makes me sound like a slop artist,

but I I promise I'm not, but I actually

have my agents now um automerging PRs

for me. Uh which is like a wild thing to

say, but um like I woke up today and

there were like 20 PRs landed and I just

reviewed them on Maine like they were

already landed and they were good. Uh so

how did I get to that point is basically

what I wanted to talk about today.

Uh, and again like yeah, feel free to

jump in if you have questions, Colin.

Um, but uh, oh yeah, of course I got to

show this this chart. Uh, where uh,

no do not trust to someone requested to

control my computer. Uh, probably won't

do that. Uh but yeah, so this chart I

think I I'm sharing this chart not to

kind of like flex but to kind of show

like the journey like so you can see

like the curve like it sort of like

inversely matches the contributions I've

been able to land at cursor. So I joined

five months ago and five months ago like

I you know my first month I was like not

very productive because I was ob you

know I was learning the codebase didn't

know what the heck was going on and as I

got more confident in in my agents uh

I've really been able to kind of ramp up

my productivity. Uh and again like yeah

like last month I shipped a thousand PRs

which is ridiculous. Uh, and then this

month we're only on the 12th. I'm

already at like almost 800 PRs landed.

Uh, so the velocity is definitely high

and you you I'm sure a lot of you will

definitely be questioning like how how

much of this code is actually good. Um,

and I think yeah, like that's definitely

fair to question.

Um, but uh, yeah, I think I think if you

set up your agents well, you can

definitely get to a very similar level.

Um, and so I'm going to talk about how

we do that.

Uh so for me I think I'm curious like I

guess call in your experience as well

but uh for me I think the most important

skill that you should have in your

toolbox when you work with agents is

verification.

Uh and by verification I mean the

ability for an agent to actually run the

code uh or take CPU traces or heap

snapshots or uh you know open an iOS

simulator whatever you know however your

application is exposed to your users it

can do the same thing and uh run it for

real and actually test and verify it

don't work because that's the thing that

really closes the loop. Uh it doesn't

guarantee your agent writes good code uh

but it allows them to at least write

correct code uh which is a big a really

big step forward for being able to trust

your agents. Um

I will I can share one example that we

have uh within cursor. Uh

oops

where let me open this.

Let's make this uh make me full screen.

There you go.

Uh so for the for cursor's agent window

uh so this is actually an interesting

story but uh when I joined cursor 5

months ago uh they're actually uh well I

was supposed to join a different team I

was supposed to join like the cloud

agents team uh but then since I have a

lot of experience working on react and

agents window is a react application uh

I was

uh I was asked to basically help out

with uh the agent window work. Uh but um

there wasn't really a lot of like skills

to help me. So I just found myself like

okay uh agent window is going to launch

in like a week, right? we have a really

tight deadline and um there was uh you

know I was just sitting there like okay

I'm going to open up the performant the

the Chrome dev tools and just like take

a trace look at it myself and try to

make sense of this flame graph and keep

in mind I was just like in my first week

so I had no idea what I was looking at

no idea what you know I mean I had some

idea but you know the codebase was

completely fresh to me uh and I realized

like my agent had no idea either you

know like I would take a screenshot of

the tree. I would download trades. I

would send it to it and it be like,

"Yeah, it kind of looks like this, you

know, uh, and it would like confidently

state like it's this thing." And then

I'd try to fix that. And turns out

that's not the actual thing.

So, this was very very slow process. And

if you've ever done any like performance

work yourself or, you know, just even

development with an agent where you

don't have a verification skill, you are

the verifier, right? You you're the

bottleneck. you you you tell your agent

to do something and then it goes off and

write some code. Then you open up your

you know local dev build and then you

start to say oh you know doesn't work.

Then you got to copy paste screen uh you

know screenshots or console errors or

whatever. Uh and then your agent like

slowly kind of like uh you know works

with that and then tries to understand

it and um fix the thing but then you're

constantly just in the loop and and

being a bottleneck. So there's really no

way to parallelize. So the control glass

skills are like one of the first skills

I built uh for cursor. Uh and glass by

the way is the code name for agents

window that we use internally but it's

just cursor I guess. Um and so this

skill uh is I guess the the the code

itself is not super interesting. Your

agent can very easily make one for you.

uh where if if you're building an

Electron app or a web app or even iOS uh

applications uh you can teach your agent

how to use like the ChromeDev tools

protocol or through uh Apple has some

utilities as well for running the

simulator and taking traces and

controlling programmatic control as

well. Uh so that's really useful. Uh but

one thing I want to talk about is uh the

this thing

uh where is the read me? Uh so this

skill comes with this very unique

feature called or not feature uh unique

file called a feature map. And so the

story then is like I built this skill

and so now the agent was able to uh

actually run the agent window and take

traces and whatnot. Uh but it had no

idea what what the agents window was. So

um you know like someone would say like

oh the the left sidebar is like laggy or

something like that or you know the

right side the the PR tab is not working

and the agent would just be like kind of

flailing around. It would spend a lot of

time trying to like look up the code and

you know where is this feature? How do I

actually get to it on the UI which made

it basically completely useless. uh you

know like we would I would run this

skill locally and you know it would

spawn a dev build uh but then it' just

be churning like I just try to click

here it wouldn't know how to get to

things um and it was just an awful

experience uh so is putting arrows on my

screen um so uh yeah this this feature

map has been really useful uh because it

teaches the agent how to get to all of

the features that you have. Um, and in

PAC, the plugin that I I've made, uh, if

you search for PAC cursor on Google, you

you'll find it. Uh, but there is a

create verification skill in that plugin

where it actually helps you set up

something like this for yourself. Um,

including the feature map. So, it will

actually explore the code and build up

this initial feature map that tells your

agent how to get to all of the different

features that you have. Uh and this is

extremely powerful because now like you

have these user reports that come in uh

you you can actually map even like a

vague report or even a screenshot. So we

have this uh internally at cursor where

uh we have a slack channel with you know

lots of people giving us feedback on the

agents window and rockbot and whatnot.

Uh, and often times the report is very

bad like very low quality like someone

will just put very often we get like a

screenshot like and then someone just

says question mark question mark

question mark like what is this and you

know like without this your agent is

like I have no clue right but with a

feature map like this it has a lot more

context and understanding of how to

actually navigate how to get to all the

different features. Uh so like you know

example like I guess like the sidebar

like what is the sidebar uh you know

like all the different sub features that

are present in it um like from the user

point of view here's where how do I get

to it all the different keyboard

shortcuts

uh even like the the what do you call it

the DOM elements or yeah like the

attributes that you use for selecting

things through the CDP uh are all there.

So uh again yeah this is like really

really powerful uh for for agents

>> uh and pstack ships uh that create

verification skill but also a maintain

verification skill uh so you can keep

this up to date.

>> Cool. Yeah, I was just going to ask how

you created that. So do you mind sharing

a little bit more about um that that

process in the context of Pstack and

maybe just what Pstack is for the folks

who aren't familiar?

Yeah. So, P stack is pretty interesting

because uh well, first of all, the name

is kind of goofy. like the P the P and P

sack is like potato potato snack because

I um so uh uh there is a pretty uh

famous person Gary Tan who is the CEO of

Y Combinator and he's come up with this

plugin called Gstack uh Gary Stack and

uh funnily enough we share the last name

we have no relations uh but I thought it

would be funny to kind of you know poke

fun at Gary and make peace stack my

version of of of of [laughter] his

plugin. Uh but kind of tailor it to my

own set of p engineering practices.

Uh but I honestly actually never set out

to build PAC. Uh it just started with a

bunch of skills, right? Like I started

with that control glass skill and then I

started with another skill like called

how which I also noticed through like

observing agents. Um, so like you know

in the early days of me, you know,

trying to climb this ladder, I was like

super in the loop and I was basically

nitpicking my agents to an extreme

degree. I was like I would tell it um,

you know, this feature has stopped

working. Here's a bug report. Like why

isn't it working?

And very often the agent would just like

confidently state like, "Oh, it has to

be this, right? It has to be this

thing." And I noticed like when I looked

at the actual tool calls, I noticed it

wasn't actually reading the code that I

thought should be affected. And that

made me just extremely suspicious. And

at that point, I was like, I'm not going

to I can't trust any this agent anymore

cuz it's just it's just completely

hallucinating. And I think

I think it's very easy to just you know

like build up that distrust and not and

kind of feel helpless like you know you

don't know how to help your agents

succeed. But like again I think the the

the management analogy is super helpful

because like imagine if you were a

manager of an engineering team and you

had an engineer on your team who was a

really good coder. No business context

whatsoever. you know they they just you

just hired them and they they onboarded

you know like five seconds ago. Uh and

so how do you actually teach that person

to be effective? So how you do that is

through a skill. uh a skill being just

you know it's just markdown right but

you know it encodes a lot of information

instructions a lot of uh you can really

draw out a lot of intelligence from an

agent by well some people on Twitter

call it like you know pull the agent to

a different latent space which is kind

of like a fancy way of just saying like

since uh you LLMs are sort of like they

predict the next token uh when you give

it some high quality tokens uh to to

begin with then you know it it can kind

of pattern match on like a higher space

that's you know smarter

um so that's like a very interesting

model there but yeah I built Pstack very

very incrementally uh so uh started with

just really observing how agents you

know all the fail different failure

modes of of that agents were having and

every time I saw that I just okay I'm

just going to make that a skill right

like stop hallucinating actually go and

search up, look up the code, use a lot

of sub agents, uh, and yeah, stop

guessing.

>> Yeah, that makes sense. One, one kind of

followup question here, uh, both from

myself and from a bunch of people in the

chat. So,

>> I guess it's two two parts. So, one is

like how do you maintain these skills?

So, like the product changes over time.

Obviously, there's a lot of people who

are shipping against the codebase. So,

how do these skills get maintained? Uh

and then second to that is like how do

you know when your verification is is

good enough? Uh like and you know you

can trust that the ver verification

loops that you've built are going to I

guess you trust that the outputs uh when

they're done.

>> Uh yeah maybe I'll talk about um I think

there are some related maybe I'll start

with this one first. So like how do I

maintain these skills?

So, um, if you're not familiar with this

concept, an eval [clears throat] is

essentially like a way to, uh, well, I

the mental model I have is like it's

like a unit test for an agent. Um, and,

uh, you can actually make your own

evals. You don't need like a special

framework for them. You can build you

can you can build one depending on like,

you know, how scientific and how

rigorous you want to be. Uh, my screen

is red.

>> Yeah, there's a little button. Um,

sorry. Do

>> you mind like disabling the drawing or

something? I I can't see my screen.

>> Yeah, sorry. If you guys could not draw

on the screen, that'd be great. But, um,

there's a little button in the

>> troll.

>> Yeah, the little drop down.

>> How do I clear?

>> Yeah, you got it. Perfect.

>> Yeah. So, eval

test your skills basically. And actually

in Pstack we ship uh under potato mode

there's a playbook if you search for it

called eval playbook. Um and it's uh

uh it's like not it's actually pretty

pretty rigorous the way it's done. Uh

but um essentially what I do is I spawn

a lot of different sub agents. I have

like my main coordinator agent uh come

up with a rubric for uh what I want the

skill to do. Um, and then it spawns all

these sub aents and it's it creates

individual directories for them, uh,

which are cleverly named to not let the

sub agent know that it's being evaluated

because, uh, agents can actually tell

and when they do, they change their

behavior. Uh, but it does a bunch of

stuff like that to um essentially yeah

like test whether or not the skill I'm

making or changing is actually doing

what I think it does. Um, and one of the

really nice things about cursor is that

we are we we support so many different

models. So you can actually eval your

skill across all sorts of different

models um and you know get a sense of

how well it performs across that

different matrix. um especially for the

models that you use.

Uh so I do this a lot. Every time I I

modify a skill, I will run one of these

uh like the EBAL playbook uh and make

sure that you know it's actually leading

to a result I want. Uh but I will say

like

maintaining skills is actually pretty

hard. Uh it requires I think a lot of

taste and observation. So, you kind of

need to be very good at being a backseat

driver, you know what I mean? Like if

you do pair, if you ever done pair

programming, for example, uh, and you

watch a co-orker code and you just like

you, you could probably do this better,

you know, you could do, you know, like

why did you not do this, right? You you

ask a lot of questions to your coworker.

And it's kind of a similar thing here.

You like you don't want to just be a

passive observer of your agent. You want

to be very in the driver seat in the

initial stages when you're building up

your own set of skills. uh you know

obviously you can use something like PAC

but if you're building your own set of

skills it's very I think you know

opening up the all the tool calls and

like reading the code and reading all

the uh the agent behavior and their

thinking blocks is a really great way to

see where they they fail right like what

what you know where are they being done

and then you can go and build a skill

for that and then with verification how

you trust it is it's I think it's also a

very similar iteration loop uh where you

know like I actually did the same

process for verifying the verification

skill where I actually get um so one

thing that's interesting about eval is

that you can sort of hill climb them

meaning that uh your eval can produce a

score right uh a score that you can get

your coordinator to produce uh but also

comp uh you can have a judge agent of a

different model to uh kind cross

reference and make sure that the first

model is not being biased, right? The

model that's judging all of the sub

aents that are running the thing. Uh but

you can also like hill climb. So meaning

that you can you can use like /loop in

cursor and you can say okay keep looping

on this eval right until everything is

10 out of 10 as an example. Uh and I did

the same the basically the same approach

with the control skill. And so I kind of

it was very it was very hands-off

actually. Uh so you know I uh I kind of

built I built that skill that way like

the CLI in that skill. Um and over time

it's gotten really good. Uh but yeah it

was definitely not super smooth at the

beginning. It required a lot of

iteration and I think there's an analogy

here for me which is um well I make this

analogy later in a different slide on my

drawing here. Uh but I think of it like

uh you know as a as a engineer now

you're sort of more like you uh like

maybe a manager or the analogy I like is

like you're like a a chef in a

restaurant. Uh you you're the head chef.

Uh you're not cooking all the food

yourself anymore. You have a team of

cooks, right? You have line cooks, you

have a sue chef, you have, you know, all

these different stations.

Um and it's your job to really design

the environment. you know, you you're in

charge of setting up the kitchen. You're

in charge of, you know, like giving

tasks to different people. So,

um yeah, it's a very interesting way of

working. Uh but yeah, that's that's how

I've basically built uh these

verification skills.

>> Yeah, just just one followup there on

like to go try to go one layer deeper.

So, are you let's say we wanted to build

um an eval or a skill for for something

and we wanted to kind of get better on

its own, which is is what I think you're

suggesting. Uh are you doing that in

like a work tree kind of isolated with

like the sub agents and and then the

reviewer agent and and all that? Is it

happening like in some type of cloud

hosted environment? Like what's the the

more the practical steps? If I wanted to

go do this uh and like set up a

verification system for something, what

would I what would I do or where would I

start?

Um I think that uh the best place to

start is local because you can observe

you can definitely observe what your

agents are doing. So, uh, if you're

building a verification skill for

yourself, uh, I would definitely start

local and just have your agent bring up

the application, whether it's like a CLI

or, uh, desktop app or whatever. And so,

you can actually observe, right? You can

see how the agent is interacting with

the the application. You can see it, you

know, how it calls like the different

APIs that that allow it to interact with

the uh the application.

Um but uh for me personally uh I have

basically been kind of all in mostly all

in on cloud agents because they're

extremely powerful. Uh and the really

powerful thing about cursor is the the

cloud agents actually where if you spend

a little bit of time setting up your

environment

these control skills these verification

skills pay a huge amount of dividends

because it's not just something that

makes you as a single engineer better.

It actually levels up your whole team uh

and even your whole company because uh

you can actually start thinking about

cloud agents. start thinking about

automations that automatically do things

like uh I I get I I kind of talk about

this a bit later, but I'll just kind of

get into it. Uh where where you know,

for example, like I talk a lot about

this agent we have called Benny, right?

who who uh you know takes all of the bug

reports that we get and it automatically

goes off in the cloud, opens up a cloud

uh it's you know its desktop. It runs

cursor in its own computer and it uses

the same control skills to interact with

the application and try to reproduce the

bug uh or the user report, right? And

this is so so powerful because at once I

can immediately I I get so much

information from this automatically like

here in this example you can see that uh

the Benny actually reproduced the bug uh

but it's already fixed on main. So it

actually confirms that we fixed this

problem already and all I need to do is

just release another build of of cursor.

Uh, so that's like huge information

there that I didn't have to go off and

sit with an agent, you know, and spend

an hour trying to figure like is this

fixed, is this not fixed. So you you you

gain back so much time. Uh, but you

know, everybody on my team benefits from

this. Everybody at the company benefits

from this. Uh, so definitely think that

uh, you know, keeping these uh, using

cloud agents is super powerful. Uh, but

yeah, it's like a journey. You have to

trust it first, right? before you you

get to this point. And that's it goes

back to what I was saying here where,

you know, it's very hard. It's almost

impossible. And I would definitely

encourage you not to try to jump from,

you know, like if you're still in this

zone, you don't want to jump to like I'm

going to spawn a hundred of thousand or

thousands of cloud agents right now

because you're just going to waste a lot

of tokens. Um, and it's going to be

extremely expensive.

>> Yeah. So just to kind of recap so far,

basically the if we wanted to go on the

journey that you've kind of gone on, it

would be to start with verification,

building some some skills and some some

ways of determining that the agents are

producing at least like correct code.

Whether like you said whether it's good

code or not is maybe a separate

question, but like it's it's technically

solving the problem by looking at you

know stack traces, looking at you know

the the actual behavior in the app and

so on. Um, and then once we trust it

locally, then we can start to think

about scaling into the cloud and running

more agents that are picking up signals,

I guess, on their own, right? So whether

that's like a bug report that comes in

or something, they can go and pick it up

and solve the problem and and give us

back a PR. And then maybe the last step

is like automerging the PRs, which uh is

where you're at, maybe not where

everyone is at.

>> Um, and then and reviewing them on main,

but um, is that is that about right?

>> Yeah, exactly. I think yeah, that's why

I drew this this uh this this curve,

right? because that this this basically

describes my journey of you know when I

started barely could use a couple agents

and I was just observing every single

thing. I think there's really no

shortcut for going from here to there

because this is really about your

personal level of trust in agents,

right? Um obviously, you know, as a as

engineer, you don't want to just slop

code into production. So, how do you

actually build up that trust? Takes um a

lot of uh I guess taste and judgment. Um

but, uh you know, like I think plugins

like Pstack definitely kind of help you

uh get up to speed much quicker. Uh, and

so I guess it's like if you trust me and

you trust Pstack, then in by extension

you can maybe trust your agents. But if

you don't trust me and I I definitely

would not encourage people to blindly

trust me. Uh, uh, you know, if you build

up your own set of skills that you can

obviously, you know, take a look at PAC

and kind of fork it, make it your own,

improve the skills. Definitely encourage

that. Uh but for me it's really all

about it just keeps coming back to

trust. You know every one of us here in

this chat have a different standard for

engineering. Uh and there are different

things that are important for us in our

codebase. And uh when you are able to

encode all of that into skills and you

can verify that your agent is actually

doing them that allows you to really

kind of ascend this curve and um uh you

know start automating things. Uh there's

another piece I wanted to talk about. Um

if there's more

>> Yeah, go for it. I'll I'll pick up more

questions as I go. But

>> yeah, I think there's a third part to

this which I haven't talked about yet,

which is kind of an interesting one

which is like refactoring and rewriting

like one of the uh I guess most

controversial one of the most

controversial topics in the industry I

think is like should you rewrite your

app or not? Um because I think engineers

are very prone to this where especially

when you join a company you come in and

you see like the code base and you're

like oh man this is like who wrote

this code you know it's terrible I want

to rewrite the whole thing there is a

very common inclination and I think a

lot of you know before agents um and I

guess arguably even now people will

definitely discourage you from re

rewriting stuff but I'm actually here to

make a case for why you might want to

consider

Um because

I think it really depends. Uh you know

uh brownfield applications I think are

actually in a pretty good spot

especially if they're set up well

already. Uh and like recently I've been

talking to some people but uh you know I

I was just observing I I just noticed

this parallel which is that a lot of big

tech company problems are now

everybody's problems. Um, and the big

tech company problem, you know, like

when I was working at Meta, like we had

this giant monora repo, we had like, I

don't know, tens of thousands of

engineers just, you know, like banging

on their keyboards and and shipping code

and

a lot of really great engineers at Meta.

Uh, but, uh, I'll say like, you know,

you'll be surprised that the code

quality is actually not that good.

[laughter] Um, and so I often joke that

like, you know, before AI sloth, we had

human sloth. Um and so uh you know I

think a lot of big tech infra like uh

like what Meta has or Google you know

you know really big tech companies are

actually designed for that where you

you're sort of like you're catering to

the the you know like uh this sounds so

bad to say but like the the least

capable engineer on your team right you

build you build frameworks you build

conventions you build guard rails you

know you restrict credentials so that

you know your intern doesn't wipe your

production database

Um

there's uh you know if you have that

level of infra already I think your

agents can actually already do a very

solid job right because they have the

the guard rails are already in place for

agents to not cause havoc or not cause

too much havoc uh in your codebase um

and you can always add more you know

guard rails. Uh but I think like green

field applications especially are you

know like the brand new applications are

like the biggest risk in my opinion. Uh

and also the greatest opportunity

because you know if you vibe code a

project uh a prototype um like we did

for Grockbot you know Grockbot was spun

up very very very quickly. Um and if you

if you haven't heard of of Grockbot it's

like a a new application we just

launched yesterday. Uh it's it's really

cool. uh lets you orchestrate your

create like individual agents that have

their own identity and you can kind of

orchestrate them. It's super cool.

Definitely check it out. Um but yeah,

that was it's like a very it was a very

green field application like most

prototypes are so like vibe coded very

quickly. Humans were not reading the

code at all. And uh I had this tweet

recently uh where I said something about

organic architecture. Um

maybe I'll find it. Uh but the idea is

that

uh when you have a completely vibe coded

application, you essentially have no

guard rails whatsoever. So uh your

agents

when you give them a task, they will

just solve it in whatever method is the

most convenient. And over time you get

into this uh situation where you have a

codebase that is spiraling out of

control because you don't understand it.

Uh your agents understand it I guess in

a way but like they've built something

that is you know optimized for short for

shortcuts. Uh and uh you know it will

you will suffer you'll have a lot of of

issues with that application.

Uh so I think starting your codebase

with uh like very strong constraints is

very much needed uh because like when

you have a codebase that you can trust,

right? when you have guardrails that

actually help you uh uh help your agents

write good code, you can get into the

you know like into this part of the

curve where I I where like I I said you

know I woke up today and I had like 20

PRs merged u by my agents and that's

because I invested a lot a lot of time

uh over 600 PRs I I I calculated

yesterday uh when I refactored all of

Grockbot to this new architecture that

I've been

Um,

and yeah, I've gotten to a point where I

I don't really look I really don't look

at the code anymore. And um, I say that

not just, you know, to sell you tokens,

but because I, you know, it it it took a

lot of work to get to that point. I

spent a lot of tokens to get the

codebase to this point where I no longer

have to look at it. Uh but I'm very

excited because you know of the

potential where you know it's not just

this doesn't just benefit me it benefits

everyone contributing to Grockbot and it

also empowers you know designers and

product managers and you know pe uh even

GTM people to add features to Grockbot

and I don't have to worry you know I

don't have to to wake up at night in in

the middle of the night and worry like

oh someone's just merged a perf

regression right I have a ton of

constraints and CI is like it's actually

very annoying to write code in in graph

web but like agents absorb all of that

annoyance.

Um but yeah I'm happy to talk about what

exactly that is. Um

>> yeah I think one question um

>> before we get into the this part here is

just around that element of like what

your your your CI looks like or maybe

some of the constraints and then also

like the average PR size. I saw a

question about that earlier just to give

people you know kind of a a glance. So

it doesn't have to be like

mathematically average, but just uh you

know like what generally the size of the

a PR is. Um if it's only a couple lines

of code or you know um yeah

>> um

I think it depends uh

I'm trying to do this in a way where I'm

not going to like

>> you. Yeah, you don't have to share the

actual number like an actual average. I

think this this is fine

>> benchmark

>> but like we have so okay this is not

that interesting but uh well fun fact is

that virtualization in grockbot and in

uh cursor is actually powered by uh

pretext uh which is a sort of new

library that someone's built um that's

really interesting you should you should

check it out but that's not really that

important uh I think the average PR size

I actually don't know. I pro I don't

know if I want to click on these. Uh I

probably can, but I would say like they

can range anywhere from a few hundred

lines or 50 lines to like a thousand

depending on what the thing is doing. Uh

so like here I'm actually like deleting

a bunch of files. So I expect that it's

just like mostly deletion. Uh but yeah,

it kind of varies.

>> There's no like Yeah,

>> there's no like hard cap or hard limit.

Are there all like 50 line P?

>> There's no hard cap. Yeah, there's

definitely no hard cap, but I I do

encourage my agents to split up their

work into multiple PRs. Uh I do that

mostly because uh I like I like the idea

of the I guess maybe this is much harder

to do now as in the world of agents and

you have like so many commits, but I

like the idea that you know the git

history is a very rich source of

context. Uh, and I like the I like each

PR to sort of atomically describe what

that small piece of thing is doing,

which also makes it easier for me to

revert changes and like figure out, you

know, oh, I shipped a bug and it's just

it's here, right? It's not in this

40,000 line PR where who knows what

landed in there.

Uh, but yeah, I don't have a hard cap on

PR size.

>> Cool. And then um yeah, also quick

question on like CI. So again, you don't

have to go into like uh the screen share

of like your CI does, but just generally

would you describe what the CI kind of

looks like uh or how strict it is?

>> Uh yeah, so

uh well specifically for Grockbot. So

Dune is the is the sort of cheeky code

code name for the architecture that

we've built for Grockbot. Um the CI

looks pretty annoying because there's

checks for everything. So like literally

I have um uh well if you've written any

React for example you know you know that

one of the biggest foot guns in React is

use effect. Uh so in

uh Dune and in graphbot we've banned use

effect. So Dune is just you can the the

mental model of what Dune is uh you can

kind of think of it as like Nex.js JS

for uh electron apps and it's designed

for agents to write uh and it's like

custom for you know our agent powered

applications. Um so the CI checks are

very like specific to that like you know

don't use use effect it's it's it's

banned like CI will fail uh and yell at

you. We have like some of the more

interesting ones that people might raise

eyebrows is like I actually ban code

comments as well. Uh which is very

interesting. Uh but I've noticed that

99% of the time agents just write code

comments that kind of describe some

historical thing that is actually

totally irrelevant to the code. Um like

it will often say like you know oh

Lauren said you should never do this and

it's now in in a code comment. I'm like

what? like why what that was I didn't

say that as like a durable you know

global rule I just meant like your this

PR sucks and you should change that part

but agents don't really understand us

that well surprisingly uh and or they

kind of assume too much and they kind of

do things in like very stupid ways so

like yeah we just ban everything

everything you can imagine like the

agents are bad at we ban uh so one

example that we actually suffer for a

lot in the agents window is we have uh

you know you if you've used the agents

window you've definitely seen

performance issues and you know we're

constantly trying to fix them. Uh but

it's like a it's a never- ending

struggle because there's so many pull

requests that get merged every any one

of them could just regress performance

or stability or reliability. Uh you know

the agents window doesn't have this

architecture yet. I plan to do bring

this learning back there and kind of

refactor everything there. Uh but uh it

just regresses super often uh because uh

there's just one example is like we have

very poor um isolation between

processes. So like on you know on on

Electron you have a renderer thread that

renders your UI but you also have like a

main thread that you can run other code

that you know doesn't need to block the

renderer.

Um, but we do a poor job of separating

those things. And so oftentimes you just

accidentally have code that gets pulled

into running on the renderer thread and

then all of a sudden you're competing

with the the renderer that you know that

has a very if you want like 60 fps you

have to every frame that gets drawn has

to be done in 16 milliseconds. So very

very small you know deadline per frame

uh if you want you know a very smooth

product. Uh and when you start building

bringing in accidentally bringing in you

know things that are like very

computationally heavy or they have a lot

of IO uh then you just get into like a

lot of jank right your your FPS really

drops you start uh you know losing

frames you get long tasks that take more

than 16 milliseconds and you just get

this really choppy experience.

So all of those patterns that we've

learned basically building electron apps

we've encoded into this framework and it

becomes like a hard failure. So I

literally in in grabbot we literally

have a directory called electron main

electron renderer and we have uh import

uh CI guess where we actually check the

dependency graph to make sure you're not

accidentally importing code from one

directory to another. Uh so that's

enforced by CI um as well as bug bots uh

which is our which cursors um like code

review tool that runs on CI uh you know

in our agents MD it's everywhere like so

I I I um I have this thing here where I

I talk about like um you know like there

are multiple layers I think for building

a good codebase. Uh obviously the

codebase is one where uh if you have an

architecture like this where it's

extremely strict uh you know the the the

way to build features is very

conventional that's like the strongest

strongest level of enforcement because

agents just love to copy existing

patterns. So uh one example of this in

rockbot is like we have this these

concepts called like a feature and we

have entry points and transcript cards

like oh you know the cards that you see

in the chat these are all like like

nouns I guess in in the framework and so

there's a very conventional way of

creating them and so like a feature is

all in in a single directory as an

example and so all of the code that

contributes to that feature lives in one

directory so it's all collocated in one

place. Makes it super easy. You know,

agents don't have to like uh grap around

and try to figure out like where all the

things are. It just looks at the feature

and like, oh, okay, I'm working on the

onboarding feature in Grockbot. Uh I'm

just going to work in this directory.

And for 80% of the work, it's mostly

just very encapsulated there. But, uh

like it's like very it's like designed

again for you know like the dumbest

agent like you don't have to think,

right? the the the one of the key

principles I have for this framework is

like the shortest the shortest path is

the best path.

So uh because that plays exactly to how

agents love to write code is like they

like to take shortcuts really you know

they they'll find the quickest way to

solve the problem. So why not make that

the best way to solve the problem? Uh so

I I I probably won't get into all the

specific details. Um and uh the the this

framework is really more of a collection

of ideas and principles rather than

something that will open source. Uh you

can you can you know screenshot this I

guess if you want and uh tell your agent

to uh do some build build something like

this for you too.

Um yeah, but it's really all about the

layers uh you know like the the codebase

is one part with features uh and

directories and you know import or

blocking import dependencies uh that

shouldn't be imported. Uh but and and it

all enforces that and static analysis.

So like uh there's CI checks, we have a

lot of lints for bad patterns that we

observe. uh compiler diagnostics

uh there's also rules and bugbot which

are um I think like three four five are

more soft right these two actually make

make CI red right so that you know

there's a hard constraint where the

agent can't just write crappy code

for rules and skills and buggbot your

agents can still forget right you can

still or it may not always consistently

apply them. So, I like to layer them,

but I don't I don't like to rely on them

as the only source of enforcement

because it's very very soft, right? And

if you if you only have rules and bug

ball and skills and a style guide for

your code, you will it's only a matter

of time before your code base looks like

complete trash. I'm sorry to say that,

but uh I definitely recommend yeah like

you know investing in you know things

that can be hard and forced, right?

Okay. And this is why you know maybe uh

the choice of text stack that you use is

also very important. Um like I think for

example Rust is sort of making you know

it's like getting super popular again.

Uh because the compiler is so strict

right the compiler enforces so many

different things. you know, there's a

borrow checker that you have to appease

and if as long as you make sure your

agents don't write unsafe code blocks,

uh you can more or less feel somewhat

confident that if the code compiles, it

probably works and it's good. Uh but you

see it gives you that level of trust and

confidence that you as a human engineer

no longer need to go and check it

yourself. you know, you you rely on code

and static analysis to actually make

that uh a lot smoother. Um, and I I

guess the worst part, the worst place to

be in is if you are stuck in code review

land where you actually enforce all of

the constraints, the invariance in your

codebase by literally the human person

saying, you know, reading the code and

like, okay, you should not do this,

right?

Every time you have to do that, you

should consider that as a code smell

like a anti- pattern and you should say,

"Okay, instead of me commenting on the

PR, how do I turn this into a hard

rule?" Right? How do I turn this into a

lint rule? How do I turn this into a CI

failure? Or how do I even categorically

eliminate this problem uh entirely? Uh I

I can talk about another migration I've

done, but I'll probably pause here.

>> Sure.

>> Yeah. I feel like that's that's where I

am to be honest is is what you're

describing right now which is that like

I don't have all of these rules. So I

have some things to go do after this

session in terms of being able to scale

my agents. I'm I'm definitely on like

the uh you know maybe a couple of

parallel ones locally stage like two to

three locally and I'm sure most people

here are on the same. So uh yeah I know

we only have a couple minutes left.

Lauren, was there anything else that you

wanted to to highlight? I obviously

there's lots of questions so I can grab

more but I want to give you a few

minutes if there's anything else you

want to talk about.

>> Well, I think I've been yapping for

quite a lot so maybe let's just do

questions.

>> Okay, cool. Uh, one question that had uh

a couple of uh came up a couple times

was just around like token usage.

>> So the the question is like is is what

you're describing a realistic thing for

people who are on you know uh a normal

set of token usage. They don't have you

know basically unlimited tokens uh to

work with.

I think that's a really good point. I

mean, like obviously, you know, I work

at a AI lab where we have unlimited

tokens. So, uh I definitely cannot

say that, you know, this is something

everyone should do in the exact same way

that I did it. I think it's possible to

get to this point without, you know,

breaking the bank.

But you know if you're like an

engineering leader or you know you're

you you have a startup that you lead um

I think to me it's a question of ROI um

and it's like uh yes you spend a lot of

money on tokens in the upfront stage you

know like refactoring a code base is

going to take a lot of tokens uh adding

all these things uh is going to take a

bunch of tokens but if we're heading to

a world where agents are writing all the

code and you know You want to be very

lean, right? You don't want to have to

hire, you don't want to be, you don't

want to become like meta, right? Like I

mean like in terms of you don't want to

become a 10,000 person engineering or

because I mean that's a cool problem to

have, but also you you have so much

overhead. There's like planning, you

know, like you it's it's a personally I

I wouldn't uh it it's not super fun. But

um I think you want to stay very nimble,

right? And you want to you want to be

like agents are all about allowing you

to do things that you couldn't do

before. That's really to me like the

value of agents, you know, it's not just

storing tokens on every single little

thing, but um to me like the thing I

couldn't do before is like enforce this

level of constraints in a codebase by

myself, right? Like I'm just a single

person, you know? Uh it would have taken

me years to build this framework uh and

do all the refactoring and

test everything myself and verify you

know like run imagine if there it was

just me right know in in pre- agent era

just like running you know by it would

take me so long right and my salary is

pretty high right like so you know the

the question I think an engineering

leader might have is just then you know

like what is there's a trade-off of do

Do you hire someone to do this or do you

spend the tokens to set up a codebase so

that even the the most naive, right, the

dumbest agents can do a good job? And

when you actually get to this point,

like even agents that are not, you know,

fable size do an excellent job of

writing code. And this pays a lot of

dividends as well for me personally

where I've empowered not just myself but

again like PMs, designers, engineers who

are not familiar with Grockbot to just

contribute in a way that is sustainable.

So I think yeah it's definitely like a

trade-off for sure. You know like

nothing is like free for sure. Uh and

tokens are pretty expensive. Uh but oh

actually uh I I I don't know how many of

you have seen this but we actually

announced Grock 4.6 today so very

exciting finally out um so yeah graph

4.6 would be like a great it was very

very smart uh it's really good on the on

the benchmarks uh and it's the same the

tokens uh well uh I hopefully I'm not

saying this incorrectly but uh I believe

the cost per token is the same as 4.5 so

you're actually getting more

intelligence for the same cost uh I

think this is an area that cursor tries

to cursor and spaceex AI try to really

optimize for like that heredto frontier

here of you know cost versus

intelligence. Uh you know we don't

necessarily want to build the biggest

model ever because that is extremely

expensive to run. It's really about like

how do you find that sweet spot right?

You don't you don't need a giant model

but it's just super smart right and it's

not very expensive for inference.

Uh but um yeah, I think to kind of round

it up, um I think it's like a it's it's

there's a if you do your own analysis, I

feel like it's pretty positive. It it'll

be pretty positive that the ROI you get

from investing in stuff like this, uh

just empowers not just yourself, but

your whole team to be so much more

productive, right? Like imagine if you

have an army of engineers like me who

are shipping so much improvements and

and bug fixes uh you know every day

right like that is pretty exciting.

>> Cool. Uh one last question before we

wrap up. This one is for the people in

product on the on the call.

>> So let's say we do have an army of

engineers who are shipping like Lauren.

I'm just curious like how is the product

team or other functions of your company

keeping up given that like if you're

shipping so quickly have are they using

AI more to do their jobs? Like as much

as you can speak to that obviously you

don't have like you're not in that role

but just curious about how that works.

>> Um I think this is where Grothbot has

been actually exceedingly powerful. uh

where so before graphbot like you know

uh obviously cursor only had cursor like

we only had agents window we had a CLI

we had an IDE and these are really like

power user tools right like de they're

designed for developers so it's very

very developerentric you can do

knowledge work in them but it like the

UI is not really optimized for that so

we actually didn't really have uh well I

think like a lot of people like you know

GTM product like they might have used

cursor uh to do their work but it

definitely wasn't like a delightful

experience for them. Um I think now with

Grogbot

uh it's become Grogbot is basically like

the kusher moment for people who are not

in tech in my opinion like it's like

it's like a very very accessible way to

use agents in a very comfortable very

familiar interface. It looks like

iMessage. Um, and it's very fun to, you

know, you can give your agent a fun

name. Uh, you can have you can kind of

do orchestration with in a very like

natural way where you can sort of, you

know, each agent's like a person and now

you got a team of agents like working on

you have one one agent per account that

you manage as an example. Or if you're a

PM, you have, you know, you can have an

agent that summarizes all the work that

Lauren did last night and then now you

know what I did, right? So I think our

PMs are leveraging that a lot and

they're shipping code too. Uh so you

know like often times they will just say

oh here's a bug I fixed it can you look

at it and then I'll go review it and

actually it's just perfect. I'm like

okay stamp. Uh so uh that I think that

shows that you know the the Dune

architecture is holding up right the all

the the really strict constraints allow

people who are not experts in

engineering to contribute at a high

level. Uh so I'm I feel like I'm already

seeing that pay off a lot where uh you

know designers and PMs are just able to

to to ship features directly. Um and

that just makes the Grockbot team super

fast, right? Where we can ship so

quickly. Um and we have a lot planned.

So I'm very excited uh to you know uh to

to ship more ship more stuff.

>> Yeah, that's awesome. Uh well, we are at

time. So, uh I guess Lauren, if if folks

want to support you, maybe go try out

Crockbot, try out uh 46 and uh you know,

get provide some feedback. But yeah,

this was awesome. Really appreciate you

taking the time. Uh thanks everyone for

all the messages in the chat. Lots of

good questions. I know we didn't get

through everything, but as I kind of

said at the top, way more questions than

than we could get through, but uh yeah,

really really thanks thanks for for

joining. Thanks everyone for joining and

uh hopefully you enjoyed the session.

>> Yep.

>> All right.

>> Yeah, I see it. Thank thanks for having

me. And uh if you have any more

questions yet, just DM me on Twitter.

I'll I'll open them up. I guess I'll let

the let the

>> You're going to get a lot of DMs.

>> Yeah, I'll open the updates. So yeah, DM

me. Maybe I'll do like a Twitter space

at some point. That's all for more

questions. But

>> really appreciate everyone for showing

up uh you know, taking an hour out of

your day.

>> Yeah. All right. Thanks all. I'll see

you the next one.

>> Okay. Thanks everyone. And bye.