Everyone, I'm Lauren, Lauren Tan. I

guess not many people know my last name.

Uh I am potato on Twitter.

Uh

potato with spelled with an E.

Um and I have been at Cursor for about 5

months.

Uh previously I was at Meta where I

worked on the React team, uh

specifically working on the React

compiler, uh which was a whole lot of

fun. Uh I'm still on the on the core

team and and uh contributing to open

source here and there. Uh

so that that's really nice that they

still let me do that. Uh and before Meta

I was at Netflix, uh where I was uh both

a tech lead and uh I transitioned to be

be an engineering manager

um

for about 2 years.

So I've had a I've had a lot of

experience going between engineering

management and being an individual

contributor. Uh

and I think something I've noticed

actually, which is quite interesting, is

that there are so many parallels with

you know, management skills and how to

like manage agents.

Uh and that's actually a a big part

about what I wanted to chat with you and

everybody else about today.

Um but yeah, that's that's me. Uh I do

have some like very light slides, but uh

it's not going to be um just rambling.

So let me just share my screen.

And hope that I don't leak anything.

Uh

Oh no, I need to allow permissions.

>> No worries. All good, take your time.

>> There's always tech tech tech trouble.

Uh give me 1 second to rejoin.

>> Yeah, go for it.

I see many of you already know Lauren

from the looks of the chat here. Um so

yeah, it's exciting to to get a chance

to chat with her uh and and go through

some of her her recent work. As you guys

heard, you know, a lot of recent

experience from from Netflix to to Meta

and then now over at Cursor.

Uh we're going to chat a little bit

about Grok Bot as well. So, uh that'll

be exciting. I don't know if you guys

saw that was a recent release. I think

literally maybe yesterday or the day

before

uh from the Cursor team, which is kind

of like um let's call it like agents for

everyone. You can go check it out if you

want and learn a little bit more about

the product. But um but yeah, we'll

we'll explore that a little bit today as

well.

All righty, welcome back.

And Lauren, you're just on mute there if

uh you want to hop off and mute and

we'll start chatting.

>> Yeah, sorry.

>> No worries.

>> It's 2026 and I still don't know how to

use Zoom.

>> [laughter]

>> That's all good.

>> Uh okay, so I assume you can see my

screen.

>> Yes, yeah.

>> So yeah, today

yeah, I think I think the big theme for

me as I've been using agents to write

code, and I'm sure a lot of you have had

the same experience as well, is

how do you trust it? You know,

especially if you are an engineer that's

been

writing code for a very long time, you

have

a lot of opinions and lessons that

you've learned about doing good

engineering.

And when you see agents just, you know,

winging it and, you know, guessing,

hallucinating, uh you know, confidently

stating that they found the smoking gun

uh for the hundredth time, uh but it's

actually not the real problem, you lose

a lot of trust. And when you lose when

you don't have much trust in your

agents,

I feel like you you really can't get the

most out of them.

And for me, the parallel is like with

management.

Uh so, if I'm an a manager, an

engineering manager of a team,

and I have a bunch of, you know, I have

a team of engineers

uh on my team and I don't trust them,

then the mode of operation I'm going to

be in is going to be like

micromanagement, right? I'll have to

spend a lot of time looking over my

reports' shoulders and checking that

they're doing their work well, you know,

that they're not shipping bugs to

production.

And so, I drew this chart because

uh it's not it's not a very scientific

chart, but

like this is how I imagine

myself and my journey through using

agents.

So, you know, like fast forward or back

forward

uh

or fast back

uh fast backwards like a year or so

when, you know, nobody was or not many

people were using agents to code. Uh

I think you

uh you get into this mode where you are

in very heavily in the loop with one or

several like a handful of agents.

And you find yourself just constantly

uh you know, trying to understand what

your agents are doing

uh and you're very very in the loop.

You're watching every single output. You

are sitting there prompting

um and you really can't parallelize

beyond that because you don't again you

don't have that trust, right? You can't

go to 100 agents

uh like spawn 100 agents when you don't

even trust the output of one agent.

So, over the past 5 months I feel like

I've really been able to uh like ascend

this

trust curve and now I'm at the point

where

uh I actually have This sounds kind of

scary to say this and it it makes me

sound like a slot artist, but I I

promise I'm not.

But I actually have my agents now um

auto merging PRs for me.

Uh which is like a wild thing to say,

but um like I woke up today and there

were like 20 PRs landed and I just

reviewed them on main like they were

already landed.

And they were good.

Uh so how did I get to that point? It's

basically what I wanted to talk about

today.

Uh and again like yeah, feel free to

jump in if you have questions, Colin. Um

but uh

Oh yeah, of course I got to show this

this chart.

Uh where uh

Oh do not trust to read Someone

requested to control my computer. Uh

probably won't do that. Uh but yeah, so

this chart I think

I I I'm sharing this chart not to kind

of like flex but to kind of show like

the journey. Like so you can see like

the curve like it sort of like inversely

matches the contributions I've been able

to land at cursor.

So I joined 5 months ago. And 5 months

ago like I you know, my first month I

was like not very productive cuz I was

you know, I was learning the code base.

Didn't know what the heck was going on.

And as I got more confident in in my

agents uh I've really been able to kind

of ramp up my productivity.

Uh and again like yeah, like last month

I shipped 1,000 PRs which is ridiculous.

Uh and then this month we're only on the

12th, I'm already at like almost 800 PRs

landed.

Uh so the velocity is definitely high

and you you I'm I'm sure a lot of you

will definitely be questioning like how

how much of this code is actually good.

Um and I think yeah, like that's

definitely fair to question.

Um but uh yeah, I think

I think if you

set up your agents well, you can

definitely get to a very similar level.

Um and so I'm going to talk about how we

do that.

Uh so for me I think I'm curious like I

guess calling your experience as well,

but uh for me

I think the most important skill that

you should have in your toolbox when you

work with agents is verification.

Uh and by verification I mean the

ability for an agent to actually

run the code

uh or take CPU traces or heap snapshots

or

uh

you know, open an iOS simulator,

whatever you know, however your

application is exposed to your users

it can do the same thing

and uh run it for real and actually test

and verify it'll work.

Because that's the thing that really

closes the loop. Uh it doesn't guarantee

your agent writes good code,

uh but it allows them to at least write

correct code. Uh which is a big a really

big step forward for being able to trust

your agent.

Um

I will I can share one example that we

have uh within

cursor.

Uh

oops

Where let me open this.

Let me just make this bigger.

Uh let me full screen.

There you go.

Uh

so for the for cursors agent window, uh

so this is actually an interesting

story, but uh when I joined

cursor 5 months ago

uh they're actually uh

well, I was supposed to join a different

team. I was supposed to join like the

cloud agents team. Uh but then since I

have a lot of experience working on

React and agents window is a React

application uh I was

uh

I was asked to basically help out with

uh uh, the

agent window work.

Uh, but,

um,

there wasn't really a lot of like skills

to help me. So, I just found myself

like, okay, uh, agent window is going to

launch in like a week. All right, we

have a really tight deadline.

And,

um,

there was, uh, you know, I was just

sitting there like, okay, I'm going to

open up the performance the the Chrome

DevTools and just like take a trace,

look at it myself, and try to make sense

of this flame graph. And keep in mind, I

was just like in my first week. So, I

had no idea what I was looking at. No

idea what, you know, I mean, I had some

idea, but, you know, the the code base

was completely fresh to me.

Uh, and I realized like my agent had no

idea either, you know, like I would take

a screenshot of the trial download

trace, I'll send it to him and it'd be

like, yeah, it kind of looks like this,

you know,

uh, and it would like confidently state

like it's this thing, and then I try to

fix that, and it turns out that's not

the actual thing.

So, this was very very slow process. And

if you've ever done any like performance

work yourself, or, you know, just even

development with an agent where you

don't have a verification skill,

you are the verifier. All right, you

you're the bottleneck. You you you tell

your agent to do something, and then it

goes off and writes some code, then you

open up your, you know, local dev build,

and then you start to say, oh, you know,

that doesn't work. Then you got to copy

paste screen, uh, you know, screenshots

or console errors or whatever,

uh, and then your agent like slowly kind

of like uh, you know, works with that,

and then tries to understand it, and,

um,

uh, fix the thing. But then you're

constantly just in the loop and and

being a bottleneck. So, there's really

no way to parallelize.

So, the control glass goes like one of

the first skills I built, uh, for

Cursor.

Uh, and glass, by the way, is the code

name for agent window that we use

internally, but it's just

Cursor, I guess.

Um, and so Well, skill, uh, is I guess

that that the code itself is not super

interesting. Your agent can very easily

make one for you. Uh where if if you're

building an Electron app or a web app or

even iOS uh applications, uh you can

teach your agent how to use like the

Chrome DevTools protocol or through

uh Apple has some utilities as well for

running the simulator and taking traces

and controlling programmatic control

as well.

Uh so, that's really useful.

Uh but one thing I want actually want to

talk about is

uh the

this thing.

Uh

where is the read me?

Uh so, this skill comes with this very

unique feature called or not feature yet

uh unique file called a feature map.

And so, the story then is like I built

this skill and so now the agent was able

to

uh actually run the agent window and

take traces and whatnot.

Uh

but it had no idea what

what the agent's window was. So, um you

know, like someone say like oh, the the

left side bar is like laggy or something

like that or you know, the right side

the the PR tab is not working. And the

agent would just be like kind of

flailing around. It would spend a lot of

time trying to like look up the code and

you know, where is this feature? How do

I actually get to it on the UI? Which

made it basically completely useless. Uh

you know, like we would I would run

this skill locally

and you know, it would spawn a dev

build.

Uh but then it just be churning. Like it

would just try to click here. It

wouldn't know how to get to things.

Um and it was just an awful experience.

Uh so, who was putting arrows on my

screen?

Um

so,

uh

yeah, this this feature map has has

really useful

uh because it teaches the agent how to

get to all of the features that you

have.

Um and in P-SAC, the plugin that I I've

made,

uh if you

search for P-SAC cursor on Google,

you'll find it.

Uh but there is a creative verification

skill in that plugin where it actually

helps you set up something like this for

yourself.

Um including the feature map, so it will

actually explore the code and build up

this initial feature map that tells your

agent how to get to all of the different

features that you have.

Uh and this is extremely powerful

because now that you have these user

reports that come in,

uh you you can actually map even like a

vague report or even a screenshot.

So we have this

uh internally at Cursor where uh we have

a Slack channel with, you know, lots of

people giving us feedback on the agent's

window and rock bar and whatnot.

Uh and oftentimes the report is very

bad. Like very low quality, like someone

just put

very often we get like a screenshot like

and then someone just says question mark

question mark question mark, like what

is this?

And you know, like the without this,

your agent says, "I have no clue."

Right? But with a feature map like this,

it has a lot more context and

understanding of how to actually

navigate, how to get to all of the

different features. Uh so like, you

know, example like, I guess like the

sidebar, like what is the sidebar?

Uh you know, like all the different sub

features that are present in it. Um like

from the user point of view, here's

where to how to get to it, all the

different keyboard shortcuts.

Uh even like the

the what do you call it? The DOM

elements or yeah, like the attributes

that you use for selecting things

through the CDP

uh are all there. So uh again, you know,

this is like really really powerful

uh for for agents. And uh and a piece

that ships uh that create verification

skill but also a maintain verification

skill.

Uh so you can keep this up to date.

>> Cool. Yeah, I was just going to ask how

you created that. So do you mind sharing

a little bit more about um

that that process in the context of P

stack and maybe just what P stack is for

the folks who aren't familiar?

>> Yeah, so P stack is pretty interesting

because uh

well, first of all, the name is kind of

goofy. Like the P the P in P stack is

like potato potato sack.

Uh because I um

So uh the uh there is a pretty

uh famous person Garry Tan who is the

CEO of Y Combinator and he's come up

with this plugin called G stack. Uh

Garry stack and uh

funnily enough, we share the last name.

We have no relation.

Uh but I thought it'd be funny to kind

of, you know, poke fun at Garry and make

P stack my version of of of

>> [laughter]

>> of his plugin.

Uh but kind of just tailor it to my own

set of prac engineering practices.

Uh but I honestly actually never set out

to build P stack. Uh it just started

with a bunch of skills. All right, like

I started with that control glass skill.

And then I started with another skill

like called howl which I also noticed

through like observing agents.

Um so like, you know, in the early days

of me, you know, trying to climb this

ladder, I was like super in the loop.

And I was basically nitpicking my agents

to an extreme degree. I was uh like I

would tell it um

you know, this feature has stopped

working. Here's a bug report. Like why

isn't it working?

And very often the agent would just like

confidently state like, "Oh, it has to

be this, right? It has to be this

thing."

And I noticed like when I looked at the

actual tool calls, I noticed it wasn't

actually reading the code

that I thought should be affected.

And that made me just extremely

suspicious. And at that point I was just

like, I'm not going to I can't trust any

this agent anymore because it's just

it's just completely hallucinating.

And I think

I think it's very easy to just, you

know, like build up that distrust and

not and kind of feel

helpless. Like, you know, you don't know

how to help your agents succeed.

But like again, I think the the the

management analogies super helpful

because like imagine if you were a

manager of an engineering team and you

had an engineer in your team who was a

really good coder, no business context

whatsoever, you know, they they just you

just hired them and they they onboarded,

you know, like uh 5 seconds ago.

Uh and so how do you actually teach that

person to be effective?

So how you do that is through skill. Uh

skill being just, you know, it's just

markdown, right? But you know, it it

codes a lot of information,

instructions, a lot of

uh you can really draw out a lot of

intelligence from an agent by well, some

people on Twitter call it like, you

know, pull the agent to a different

latent space. Which is kind of like a

fancy way of just saying, like since uh

you know, LLMs are sort of like they

predict the next token.

Uh when you give it some high-quality

tokens uh to begin with, then, you know,

it it can kind of pattern match on like

a higher space that's, you know,

smarter.

Um so that's like a very interesting

model there. But yeah, I built Pstack

very, very incrementally. Uh so uh

started with just really observing how

agents, you know, all the failed

different failure modes of of that

agents were having. And every time I saw

that, I just, okay, I'm just going to

make that a skill. All right, like stop

hallucinating, actually go and search up

look up the code, use a lot of

subagents, uh and yeah, stop guessing.

>> Yeah, that makes sense. One one kind of

follow-up question here, both for myself

and for a bunch of people in the chat.

So,

I guess it's two two parts. So, one is

like, how do you maintain these skills?

So, like the product changes over time.

Obviously, there's a lot of people who

are shipping against the code base.

So, how do these skills get maintained?

And then second to that is like, how do

you know when your verification is is

good enough? Like in you know, you can

trust that the very verification loops

that you've built are going to

I guess you trust that the outputs when

they're done.

>> Uh, yeah, maybe I'll talk about um,

I think I have something really down.

Maybe I'll start with this one first.

So, like how do I

maintain these skills?

So, um,

if you're not familiar with this

concept, an eval [clears throat] is

essentially like a way to

Uh, well, I think the mental model I

have is like it's like a unit test for

an agent.

Um, and

you can actually make your own evals.

You don't need like a special framework

for them. You can build you can you can

build one depending on like, you know,

how scientific and how rigorous you want

to be.

Uh, my screen is red.

>> Yeah, there's a little button. Um,

sorry.

>> Are you going to like disabling the

drawing or something? Like I can't see

my screen.

>> Yeah, sorry. If you guys could not draw

on the screen, that'd be great. But, um,

there's a little button in the

>> troll.

>> Yeah, the the little drop down.

>> Um,

how do I clear?

>> Yeah.

>> Okay, yeah.

>> You got it. Perfect.

>> Um,

>> Continue.

>> Yeah, so evals are a way to unit test

your skills, basically.

And actually in P stack, we ship

under potato mode, there's a playbook.

If you search for it, called eval

playbook.

Um, and it's

uh,

Uh, it's like not It's actually pretty

pretty rigorous the way it's done. Uh,

But essentially what I do is I spawn a

lot of different sub agents. I have like

my main coordinator agent

come up with a rubric for

what I want the skill to do. Um

and then it spawns all these sub agents

and it it creates individual directories

for them

which are cleverly named to not let the

sub agent know that it's being evaluated

because agents can actually tell and

when they do they change their behavior.

Uh but it does a bunch of stuff like

that to

um

essentially yeah like test whether or

not the skill I'm making or changing is

actually doing what I think it does. Um

and one of the really nice things about

cursor is that we are we have we support

so many different models. So you can

actually eval your skill across all

sorts of different models. Um and you

know get a sense of how well it performs

across that different matrix. Um

especially for the models that you use.

Uh so I do this a lot. Every time I I

modify a skill I will run one of these

like the eval playbook

and make sure that you know it's

actually leading to a result I want. Uh

but I will say like

maintaining skills is actually pretty

hard. It requires I think a lot of

taste and observation. So you kind of

need to be very good at being a backseat

driver. You know what I mean? Like if

you do pair if you've ever done pair

programming for example

and you watch a coworker code and you

just like you could probably do this

better or you know you could do you know

do you like why did you not do this?

Right? You you ask a lot of questions to

your coworker and it's kind of a similar

thing here. You like you don't want to

just be a passive observer agent. You

want to be very

in the driver seat in the initial stages

when you're building up your own set of

skills.

Uh you know, obviously you can use

something like P set, but if you're

building your own set of skills, it's

very I think you know, opening up the

all the tool calls and like reading the

code and

reading all the uh

the agent behavior and their thinking

blocks is uh a really great way to see

where they they fail, right? Like what

what

you know, where are they being done? And

then you can go and build the skill for

that.

And then with verification, how you

trust it is it's I think it's also a

very similar iteration loop uh where you

know, like I actually did the same

process for verifying the verification

skill where I actually get um

So, one thing that's interesting about

evals is that you can sort of hill climb

them, meaning that uh

your eval can produce a score, right? Uh

a score that you can get your

coordinator to produce, uh but also uh

you can have a judge agent of a

different model to uh kind of

cross-reference and make sure that the

first model is not being biased, right?

The model that's judging all of the sub

agents that are running the thing.

Uh but you can also like hill climb. So,

meaning that you can you can use like

{slash} loop in cursor

and you can say, "Okay, keep looping on

this eval, right? Until everything is 10

out of 10." As an example. Uh and I did

the same the basically the same approach

with the control skill. And so, I kind

of it was very it was very hands-off

actually.

Uh so, you know, I uh I kind of built I

built that skill that way, like the CLI

in that skill. Um and over time

it's gotten really good. Uh but yeah, it

was definitely not super smooth

at the beginning. It required a lot of

iteration. And I think there's an

analogy here for me, which is um

Well, I make this analogy later in a

different slide on my drawing here,

uh, but I think of it like

uh, you know, as a as a

engineer now, you're sort of more like

you

like maybe a manager or the analogy I

like is like you're like a a chef in a

restaurant. Uh, you you're the head

chef. Uh, you're not cooking all the

food yourself anymore. You have a team

of cooks, right? You have a line cooks,

you have a sous chef, you have you know,

all these different stations.

Um, and it's your job to really design

the environment. You know, you you

you're in charge of setting up the

kitchen. You're in charge of, you know,

like giving tasks to different people.

So,

um, yeah, it's a very interesting way of

working.

Uh, but yeah, that's that's how I

basically built uh, these verification

skills.

>> Yeah, just just one follow up there and

like to go try to go one layer deeper.

So, are you, let's say we wanted to

build um,

uh, an an eval or a skill for for

something

and we wanted to kind of get better on

its own, which is is what what I think

you're suggesting. Uh, are you doing

that in like a work tree, a kind of

isolated with like the sub agents and

and then the reviewer agents and and all

that? Is it happening like in some type

of cloud hosted environment? Like what's

the the more the practical steps? If I

wanted to go do this, uh, and like set

up a verification system for something,

what would I what would I do or where

would I start?

Uh, I think that uh, the best place to

start is local because you can observe.

You can definitely observe what your

agents are doing. So,

uh, if you're building a verification

skill for yourself,

uh, I would definitely start local and

just have your agent bring up the

application, whether it's like a CLI or

uh, desktop app or whatever.

And so, you can actually observe, right?

You can see how the agent is interacting

with the

the application. You can see it, you

know, how it calls like the different

APIs that that allow it to interact with

the

uh the application.

Um but uh for me personally, uh I have

basically been kind of all in mostly all

in on cloud agents because they're

extremely powerful.

Uh and the really powerful thing about

Cursor is the the cloud agents actually.

Where if you spend a little bit of time

setting up your environment,

these control skills, these verification

skills pay a huge amount of dividend

because it's not just something that

makes you as a single engineer better,

it actually levels up your whole team.

Uh and even your whole company because

uh you can actually start thinking about

cloud agents being that thing about

automations that automatically do things

like

uh I'll buy I get I I cannot talk about

this a bit later, but I'll just kind of

get into it. Uh where where, you know,

for example, like I talk a lot about

this agent we have called Benny, right?

Who

who uh you know, takes all of the bug

reports that we get and it automatically

goes off in the cloud, opens up a cloud

uh it's, you know, it's desktop. It runs

Cursor in its own computer

and it uses the same control skills to

interact with the application and try to

reproduce the bug

uh or the user report. All right, and

this is so so powerful because at once I

can immediately I I get so much

information from this automatically.

Like here in this example, you can see

that uh the Benny actually reproduced

the bug,

uh but it's already fixed on main.

So, it actually confirms that we fixed

this problem already. And all I need to

do is just release another build of of

Cursor.

Uh so, that's like huge information

there that I didn't have to go off and

sit with an agent, you know, and spend

an hour trying to figure out, like, is

this fixed? Is this not fixed?

So, you you you gain back so much time,

uh but, you know, everybody on my team

benefits from this. Everybody in the

company benefits from this.

Uh so,

definitely think that, uh you know,

keeping these uh using cloud agents is

super powerful.

Uh but, yeah, it's like a journey. You

have to trust it first, right? Before

you you get to this point. And that's it

goes back to what I was saying here,

where, you know, it's very hard it's

it's almost impossible, and I would

definitely encourage you not to try to

jump from,

you know, like, if you're still in this

zone, you don't want to jump to, like,

I'm going to spawn a hundred a thousand

or a thousand of cloud agents right now,

because you're just going to waste a lot

of tokens,

um and it's going to be extremely

expensive.

>> Yeah, so just to kind of recap so far,

basically the if we wanted to go on the

journey that you've kind of gone on, it

would be just start with verification,

uh building some some skills and some

some ways of determining that the agents

are producing

at least like correct code, whether like

you said, whether it's good code or not

is maybe a separate question, but like

it's it's technically solving the

problem by looking at, you know, stack

traces, looking at, you know, the the

actual behavior in the app, and so on.

Um and then once we trust it locally,

then we can start to think about scaling

into the cloud and running more agents

that are picking up signals, I guess, on

their own, right? So, whether that's

like a bug report that comes in or

something, they can go and pick it up

and solve the problem and and give us

back a PR. And then maybe the last step

is like auto-merging the PRs, which uh

is where you're at.

>> Yeah, yeah.

>> Uh and then reviewing them on main, but

um

is that is that about right?

>> Yeah, exactly. I think yeah, that's why

I drew this this uh this this curve,

right? Because that this this basically

describes my journey of, you know, when

I started

barely could use a couple agents, and I

was just observing every single thing.

I think there's really no shortcut for

going from here to there because this is

really about your personal level of

trust in agents. Right?

Obviously, you know, as a as engineer

you don't want to just slop code into

production.

So, how do you actually build up that

trust? Takes

um a lot of

uh I guess taste and judgment. Um but

uh you know, like I think plugins like

Pstack definitely kind of help you

get up to speed much quicker.

Uh and so I guess it's It's like if you

trust me and you trust Pstack, then in

by extension you can maybe trust your

agents. But if you don't trust me and I

I definitely would not encourage people

to blindly trust me.

Uh

uh you know, if you build up your own

set of skills that you can obviously,

you know, take a look at Pstack and kind

of fork it, make it your own, improve

the skills. Definitely encourage that.

Um but for me it's really all about it

just keeps coming back to trust. You

know, every one of us here in this chat

have a different standard for

engineering. Uh and there are different

things that are important for us in our

code base. And uh when you are able to

encode all of that into skills and you

can verify that your agents actually

doing them,

that allows you to really kind of ascend

this curve. And

uh you know, start automating things.

Uh there's another piece I wanted to

talk about. Um if there's

>> Yeah, go for it. I'll I'll think of more

questions as I go, but yeah.

>> Yeah, I think there's a third to this

which I haven't talked about yet, which

is

kind of an interesting one, which is

like refactoring and rewriting. Like one

of the uh I guess most controversial one

of the most controversial topics in the

industry, I think, is like should you

rewrite your app or not?

Um because I think engineers are very

prone to this where especially when you

join a company, you come in and you see

like the code base and you're like,

"Man, this is

Like who wrote this code? You know, it's

it's terrible. I want to rewrite the

whole thing. There is a very common

inclination and I think a lot of, you

know, before agents, um and I guess

arguably even now, people will

definitely discourage you from re-

rewriting stuff.

But I'm actually here to make a case for

why you might want to consider it.

Um because

I think it really depends. Uh you know,

brownfield applications I think are

actually in a pretty good spot,

especially if they're set up well

already.

Uh and like recently I've been talking

to some people, but uh you know, I I I

was just observing. I I just noticed

this

parallel, which is that a lot of big

tech company problems are now

everybody's problems.

Um and the big tech company problem, you

know, like when I was working at Meta,

like we had this giant mono repo. We had

like, I don't know, tens of thousands of

engineers just, you know, like banging

on their keyboards and and shipping

code.

And

a lot of really great engineers at Meta,

uh but uh I'll say like, you know,

you'll be surprised that the code

quality is actually not that good.

>> [laughter]

>> Um and so

I often joke that like, you know, before

AI slop, we had human slop.

Um and so uh you know, I think a lot of

big tech infra, like uh like what Meta

has or Google, you know, you know,

really big tech companies,

are actually designed for that, where

you you're sort of like, you're catering

to the the you know, like

uh this sounds so bad to say, but like

the the least capable engineer on your

team, right? You build you build

frameworks, you build conventions, you

build guardrails, you know, you restrict

credentials so that, you know, your

intern doesn't wipe your production

database.

Um

there's uh you know, if you have that

level of infra already, I think your

agents can actually already do a very

solid job. Right? Because they have the

the guardrails are already in place for

agents to not cause havoc or not cause

too much havoc in your codebase. Um and

you can always add more, you know,

guardrails.

Uh but I think like greenfield

applications especially are you know,

like the brand new applications are like

the biggest risk in my opinion. Uh and

also the greatest opportunity.

Because, you know, if you vibe code a

project uh a prototype

um like we did for Grokbot, you know,

Grokbot was spun up very very very

quickly. Um and if you if you haven't

heard of of Grokbot, it's like our a new

application we just launched yesterday.

Uh it's it's really cool.

Uh lets you orchestrate your create like

individual agents that have their own

identity and you can kind of orchestrate

them. It's super cool. Definitely check

it out.

Um but yeah, that was it's like a very

it was a very greenfield application

like most prototypes are.

So, it was like vibe coded very quickly.

Humans were not reading the code at all.

And

uh I had this tweet recently

uh where I said something about organic

architecture.

Um

Let me I'll find it.

Uh but the idea is that

uh when you have a completely vibe coded

application, you essentially have no

guardrails whatsoever. So,

uh your agents

when you give them a task, they will

just solve it in whatever method is the

most convenient.

And over time, you get into this

uh

situation where you have a codebase that

is spiraling out of control because you

don't understand it. Uh your agents

understand it, I guess, in a way, but

like they've built something that is,

you know, optimized for short for

shortcuts.

Uh and uh you know, it will you will

suffer you have a lot of of issues with

that application.

Uh so, I think starting your code base

with uh like very strong strains is

very much needed.

Uh because like when you have a code

base that you can trust, right? When you

have guardrails that actually help you

uh

uh help your agents write good code, you

can get into the you know like into this

part of the curve where I I where I like

I I said, you know, I woke up today and

I had like 20 PRs merged uh by my

agents. And that's because I invested a

lot lot a lot of time

uh over 600 PRs I I I calculated

yesterday uh

when I refactored all of GrokBot to this

new architecture that I've been

building.

Um

and yeah, I've I've gotten to a point

where I

I don't really look I really don't look

at the code anymore. And um I say that

not just, you know, to sell you tokens,

but

because I you know, it it it took a lot

of work to get to that point. I spent a

lot of tokens to get the code base to

this point where I no longer have to

look at it.

Uh but I'm very excited because

you know, of the potential where, you

know, it's not just it this doesn't just

benefit me. It benefits everyone

contributing to GrokBot.

And it also empowers, you know,

designers and product managers and, you

know,

even GTM people to add features to

GrokBot. And I don't have to worry, you

know, I don't have to to wake up at

night in in the middle of the night and

worry like, "Oh, Someone's just

merged a perf regression." Right? I have

a ton of

constraints and CIs like it's actually

very annoying to write code in in

GrokBot, but like agents absorb all of

that annoyance.

Um but yeah, I'm happy to talk about

what exactly that is. Um

>> Yeah, I think one question

um

>> Yeah.

>> before we get into the this part here is

just around that element of like what

your your your CI looks like or maybe

some of the constraints and then also

like the average PR size. I saw a

question about that earlier. Just to

give people a you know, kind of a a

glance. It doesn't have to be like

mathematically average, but just a you

know, like what generally the size of

the PR is

um if it's only a couple lines of code

or you know, um yeah.

>> Uh

um

I think it depends. Uh let me

I'm trying to do this in a way where I'm

not going to like

>> Yeah, yeah, you don't have to share the

actual number. If you like the actual

number just

>> this is is fine.

>> benchmark

>> But like we have So okay.

This is not

that interesting, but uh well, fun fact

is that virtualization in Grokbot and in

uh Cursor is actually powered by uh

Pretext,

uh which is a sort of new library that

someone's built. Um that's really

interesting. You should You should check

it out.

But that's not really that important.

Uh I think the average PR size I

actually don't know I I don't know if I

want to click on these.

Uh I probably can, but I would say like

they can range anywhere from a few

hundred lines or 50 lines to like a

thousand depending on what the thing is

doing.

Uh so like here I'm actually like

deleting a bunch of files, so I expect

that it's just this like mostly

deletion.

Uh but yeah, it it kind of varies.

>> There's no like

>> There's no like hard cap or hard limit.

They're all like 50 line PRs.

>> there's definitely no hard cap, but I I

do encourage my agents to split up their

work into multiple PRs.

Uh I do that mostly because uh

I like I like the idea of but I guess

maybe this is much harder to do now as

in the world of agents and you have like

so many commits.

But I like the idea that you know, the

Git history is a very rich source of

context. Uh and I like the I like each

PR to sort of atomically describe what

that

small piece of thing is doing,

which also makes it easier for me to

revert changes and like figure out, you

know, oh, I shipped a bug and it's this

it's here, right? It's not in this

40,000 line PR where you who knows what

landed in there.

Uh but I I don't have a hard cap on PR

size.

>> Cool. And then um yeah, also quick

question on like CI. So again, you don't

have to go into like uh the screen share

of like your your CI does, but just

generally, would you describe what the

CI kind of looks like uh or how strict

it is?

>> Uh yeah, so

uh well, specifically for Grokbot, so

Dune is the is the sort of cheeky code

code name for the architecture that

we've built for

Grokbot. Um the CI looks pretty annoying

because there's checks for everything.

So, like literally, I have um

uh well, if you've written any React for

example, you know, you know that one of

the biggest foot guns in React is use

effect. Uh so, in

uh Dune and in Grokbot, we've banned use

effect. So, Dune is just you can the the

mental model of what Dune is,

uh you can kind of think of it as like

Next.js for

uh Electron apps and it's designed for

agents to write uh and it's like custom

for, you know, our agent for the

applications.

Um so, the CI checks are very like

specific to that, like, you know, don't

use use effect. It's it's it's banned,

like, CI will fail

uh and yell at you. We have like some of

the more interesting ones that people

might raise eyebrows is like I actually

banned code comments as well,

uh which is very interesting. Um but I

noticed that 99% of the time agents just

write code comments that kind of

describe some historical thing that is

actually totally irrelevant to the code.

Um,

like it will often say like, you know,

"Oh, Lauren said you should never do

this." and it's now in the in the code

comment. I'm like, "What? Like, why what

that that was I didn't say that as like

a durable, you know, global rule. I just

meant like your this PR sucks and you

should change that part."

Uh, agents

don't really understand us that well,

surprisingly.

Uh, and or they kind of assume too much

and they kind of do things in like very

stupid ways.

So, like yeah, we just ban everything

everything you can imagine like the

agents are bad at, we ban.

Uh, so one example that we actually

suffer a lot in the agents window is

we have, uh, you know, you know, if

you've used agents window, you've

definitely seen performance issues and,

you know, we're constantly trying to fix

them.

Uh, but it's like a

it's a never-ending struggle because

there's so many pull requests that get

merged.

Every any one of them could just regress

performance or stability or reliability.

Uh, you know, the agents window doesn't

have this architecture yet. I plan to do

bring this learning back there and kind

of refactor everything there.

Uh, but

uh, it just regresses super often

uh, because uh, there's there's just one

example is like we have very poor

um, isolation between processes. So,

like on you know, on Electron, you have

a renderer thread that renders your UI.

But you also have like a main thread

that you can run other code that, you

know, doesn't need to block the

renderer.

Um, but we do a poor job of separating

those things. And so, often times you

just accidentally have code that gets

pulled into running on the renderer

thread. All right, and then all of a

sudden you're competing with the the

renderer that you know, that has a very

If you want like 60 FPS, you have to

every frame that gets drawn has to be

done in 16 milliseconds. It's a very,

very small, you know, deadline per

frame.

Uh if you want, you know, a very smooth

product. Uh and when you start building

bringing in accidentally bringing in,

you know, things that are like very

computationally heavy or they have a lot

of IO, uh

then you just get into like a lot of

jank. Uh your your FPS really drops. You

start uh you know, losing frames. You

get long tasks that

take more than 16 milliseconds and you

just get this really choppy experience.

So, all of those patterns that we've

learned basically building Electron

apps, we've encoded into this framework

and it becomes like a hard failure. So,

I literally in in Grokbot, we literally

have a directory called Electron main,

Electron renderer, and we have uh

import

uh CI, I guess, where we actually check

the dependency graph to make sure you're

not accidentally importing code from one

directory to another.

Uh so, that's enforced by CI,

um as well as Bugbot, uh which is our

which cursors

um like code review tool that runs on

CI.

Uh you know, in our agents MD, it's

everywhere. Like so, I I I I um

I have this thing here where I talk

about like

um you know, like there are multiple

layers, I think, for building a good

code base.

Uh obviously, the code base is one where

uh if you have an architecture like

this, where it's extremely strict, uh

you know, the the the way to build

features is very conventional. That's

like the strongest strongest level of

enforcement because agents just love to

copy existing patterns.

So,

uh one example of this in Grokbot is

like we have this these concepts called

like a feature and we have entry points

and transcript cards. Like all of you

know, the cards that you see in the

chat.

These are all like

like nouns, I guess, in in the

framework. And so, there's a very

conventional way of creating them. And

so, like a feature is all in in a single

directory, as an example. And so, all of

the code that contributes to that

feature lives in one directory. So, it's

all co-located in one place. Makes it

super easy, you know, agents don't have

to like uh grep around and try to figure

out like where all the things are.

It just looks at the feature and like,

"Oh, okay, I'm working on the onboarding

feature in Grokbot. Uh I'm just going to

work in this directory." And for 80% of

the work, it's mostly just very

encapsulated there.

But uh like it's like

very It's like designed again for, you

know, like the dumbest agent. Like, you

don't have to think. All right, the the

the One of the key principles I have for

this framework is like the shortest the

shortest path is the best path.

So, uh because that plays exactly to how

agents love to write code. Like, they

like to take shortcuts, really. You

know, they'll they'll find the quickest

way to solve the problem. So, why not

make that the best way to solve the

problem?

Uh so, I I I I probably won't get into

all the specific details. Um and uh the

the

this framework is really more of a

collection of ideas and principles,

rather than something that we'll open

source.

Uh you can you can, you know, screenshot

this, I guess, if you want and uh tell

your agent to

uh do some build build something like

this for you to

um Yeah, but it's really all about the

layers. Uh you know, like the the code

base is one part with features uh and

directories and you know, import

uh blocking import dependencies uh that

shouldn't be imported. Uh but and and it

all enforces that and static analysis.

So, like uh there's CI checks. We have a

lot of lints for bad patterns that we

observe. Uh compiler diagnostics. Uh,

there's also rules and bug bot which are

um, I think like three, four, five are

more soft, right? These two actually

make

make CI red, right? So, that you know,

there's a hard constraint where the

agent can just write crappy code.

For rules and skills and bug bot,

your agents can still forget, right? You

can still

or it may not always consistently apply

them. So, I like to layer them, but I

don't I don't like to rely on them as

the only source

of enforcement because it's very, very

soft, right? And if you if you only have

rules and bug bot and skills and a style

guide for your code,

you will it's only a matter of time

before your code base looks like

complete trash. Uh, I'm sorry to say

that, but uh, I definitely recommend you

all like, you know, investing

in, you know, things that can be hard

enforced. Right? And this is why,

you know, maybe uh, the choice of tech

stack that you use is also very

important.

Um,

like I think for example, Rust is sort

of making, you know, it's like getting

super popular again, uh, because the

compiler is so strict, right? The

compiler enforces so many different

things, you know, there's a borrow

checker that you have to appease.

And if as long as you make sure your

agents don't write unsafe code blocks,

uh, you can more or less feel somewhat

confident that if the code compiles, it

probably works and it's good.

Uh, but you know, you see, it gives you

that level of trust and confidence that

you as a human engineer no longer need

to go and check it yourself. You you

know, you you rely on

code and static analysis to actually

make that uh, a lot smoother.

Um, and I I guess the worst part the

worst place to be in is if you are stuck

in code review land where you actually

enforce all of the constraints, the

invariants in your code base

by literally the human person saying,

you know, reading the code and be like,

"Okay, you should not do this."

Right?

Every time you have to do that, you

should consider that as a code smell,

like a uh anti-pattern. And you should

say, "Okay, instead of me commenting on

the PR,

how do I turn this into a hard rule?

Right? How do I turn this into a lint

rule? How do I turn this into a CI

failure? Or how do I even categorically

eliminate this problem

uh entirely?

Uh

I I can talk about another migration

I've done, but I'll probably pause here.

>> Sure.

Yeah, I feel like that's that's where I

am, to be honest, is is what you're

describing right now, which is that like

I don't have all of these rules. Uh so,

I have some things to go do after this

session

in terms of being able to scale my

agents. I'm I'm definitely on like the

uh you know, maybe a couple of parallel

ones locally stage, so like two to three

locally. And I'm sure most people here

are on the same, so uh

yeah, I know we only have a couple

minutes left. Uh Lauren, was there

anything else that you wanted to to

highlight? I obviously there's lots of

questions, so I can grab more, but I

want to give you a few minutes uh if

there's anything else you want to talk

about.

>> I think I've been yapping for quite a

lot, so I'm Maybe let's just do

questions.

>> Okay, cool. Uh one question I had uh a

couple of uh came up a couple of times

was just around like token usage.

So, the the question is like is is what

you're describing a realistic thing for

people who are on, you know, uh

a a normal set of token usage, they

don't have, you know, basically

unlimited tokens uh to work with.

>> Well, I think that's a really good

point. I mean, like obviously, you know,

I work at a AI lab where we have

unlimited tokens, so uh

I definitely cannot

say that, you know, this is something

everyone

should do in the exact same way that I

did it. I think it's possible to get to

this point without, you know, breaking

the bank.

But, you know, if you're like an

engineering leader or, you know, you're

you're you have a startup that you lead,

um I think that may be a question of

ROI.

Um and it's like

uh

yes, you spend a lot of money on tokens

in the upfront stage. You know, you like

refactoring your code base is going to

take a lot of tokens. Uh adding all

these things uh is going to take a bunch

of tokens.

But, if we're heading to a world where

agents are writing all the code,

and you know, you want to be very lean,

right? You don't want to have to hire

You don't want to be You don't want to

become like meta, right? Like I mean

like in terms of You don't want to

become a 10,000-person engineering org

because

I mean, that's a cool problem to have,

but also you know, you you have so much

overhead. There's like planning, you

know, like you it it's it's uh

personally I I wouldn't uh

it it's not super fun. But, um

I think you want to stay very nimble,

right? And you want to you want to be

like agents are all about allowing you

to do things that you couldn't do

before. That's really to me like the

value of agents. You know, it's not just

throwing tokens on every single little

thing.

But, um to me like the thing I couldn't

do before is like

enforce this level of constraints in a

code base by myself.

Right? Like I'm just a single person,

you know,

uh it would have taken me years to build

this framework uh and do it all the

refactoring and

test everything myself and verify, you

know, like run Imagine if there was just

me right now in in pre-agent era, just

like running, you know, by my It would

take me so long, right? And my salary is

pretty high, right? Like So, you know,

the the question I think an engineering

leader might have is just then,

you know, like what is there's a

tradeoff of do you hire someone to do

this,

or do you spend the tokens to set up a

code base so that even

the the most naive, right, the dumbest

agents can do a good job.

And when you actually get to this point,

like even agents that are not, you know,

Fable size do an excellent job of

writing code.

And this pays a lot of dividends as well

for me and personally where I've

empowered not just myself, but

again, like PMs, designers, engineers

who are not familiar with Grok but to

just contribute in a way that

is sustainable.

So, I think yeah, it's definitely like a

trade-off for sure. You know, like

nothing's like free for sure.

Uh and tokens are pretty expensive.

Uh but oh, actually,

I don't I I I don't know how many of you

have seen this, but we actually

announced Grok 4.6 today. So, very

exciting. Finally out. Um so, yeah, Grok

4.6 would be like a great It's very very

smart. Uh it's really good on on the

benchmarks. Uh and it's the same the

tokens Uh well, uh I hopefully I'm not

saying this thing correctly, but uh I

believe the cost per token is the same

as 4.5.

So, you're actually getting more

intelligence for the same cost.

Uh I think this is an area that Cursor

tries to Cursor and SpaceX AI

try to really optimize for like that

Pareto frontier of, you know, cost

versus intelligence. Uh you know, we

don't necessarily want to build the

biggest model ever because that is

extremely expensive to run.

It's really about like how do you find

that sweet spot, right? You don't You

don't need a giant model, but it's just

super smart, right? And it's not very

expensive for inference.

Uh but um yeah, I think to kind of round

it up, um

I think it's like a it's it's there's a

If you do your own analysis, I feel like

it's pretty positive.

It It'll be pretty positive that the ROI

you get from investing in stuff like

this uh just empowers not just yourself,

but your whole team to be so much more

productive. Right? Like imagine if you

have an army of engineers like me who

are shipping so much improvements and

and bug fixes uh you know, every day.

Right? Like

that is pretty exciting.

>> Cool.

One last question before we wrap up.

This one is for the people in product on

the on the call. So, let's say we do

have an army of engineers who are

shipping like Lauren. I'm just curious

like how is the product team or other

functions of your company keeping up

given that like if you're shipping so

quickly, have have are they using AI

more to do their jobs? Like as much as

you can speak to that. I obviously you

don't have like you're not in that role,

but just curious about how that works.

>> Um I think this is where Grokbot has

been actually exceedingly powerful. Uh

where

so before Grokbot like you know,

obviously Cursor only had Cursor. Like

we only had agent window. We had a CLI.

We had an IDE.

And these are really like

power user tools, right? Like the

they're designed for developers. So,

it's very very developer-centric. You

can do knowledge work in them, but it

like the UI is not really optimized for

that.

So, we actually didn't really have uh

well, I think like a lot of people like

you know, GTM, product, like they might

have used Cursor uh to do their work,

but it definitely wasn't like a the like

full experience with them.

Um I think now with Grokbot

uh it's become Grokbot is basically like

the Cursor moment for people who are not

in tech in my opinion. Like it's like

it's like a very very accessible way to

use agents in a very comfortable, very

familiar interface. It looks like

iMessage. Um and it's very fun too, you

know, you can give your agent to be

name. Uh you can have you can kind of do

orchestration with in a very like

natural way where you can sort of

you know, each agent is like a person

and now you're on a team of agents like

working on you have one one agent per

account that you manage as an example.

Or if you're a PM, you have you know,

you can have an agent that summarizes

all the work that Lauren did last night

and then now you know what I did, right?

So, I think our PMs are leveraging that

a lot. And they're shipping code, too.

Uh, so you know, I often times they will

just say, "Oh, here's a bug. I fix it.

Can you look at it?" And then I'll go

review it and actually it's just

perfect. I'm like, "Okay, stamp." Uh, so

uh, I think that shows that you know,

the the Dune architecture is holding up,

right? The all the the really strict

constraints allow

people who are not experts in

engineering to contribute at a high

level.

Uh, so I'm I feel like I'm already

seeing that pay off a lot where uh, you

know, designers and PMs are just able to

to to ship features directly.

Um, and that just makes

the Grock bot team super fast, right?

Where we can ship so quickly.

Um,

and we have a lot planned, so I'm very

excited uh, to you know, uh,

to to ship more uh, ship more stuff.

>> Yeah, that's awesome. Uh, we are at

time, so I I guess Lauren, if if folks

want to support you, maybe go try Grock

bot, try out uh, 46 and uh, you know,

just get to get provide some feedback.

But yeah, this was awesome. Really

appreciate you taking the time. Uh,

thanks everyone for all the messages in

the chat. Uh, lots of good questions. I

know we didn't get through everything,

but uh, as I kind of said at the top,

way more questions than than we could

get through. But uh, yeah, really really

thanks thanks for for joining. Thanks

everyone for joining and uh, hopefully

you you enjoyed the session.

>> Yep.

>> All right.

>> Yeah, I see see you thanks for having me

and uh, if you have any more questions,

yeah, just DM me on Twitter. Uh, I'll

I'll open them up, I guess. I'll let the

I'll let the floodgates

>> to get you're going to get a lot of DMs.

>> Yeah, I'll open the floodgates. So,

yeah, DM me. Maybe I'll do like a

Twitter Space at some point as well for

more questions, but

really appreciate everyone for showing

up, uh you know, taking an hour out of

your day.

>> Yeah. All right. Thanks, all. I'll see

you in the next one.

>> Okay, thanks, everyone. Bye.