One area that you and the team push is

core web vitals.

>> If I'm on a page that has a bunch of

ads, I shouldn't start reading an

article and then suddenly everything

gets pushed down just because the ad is

finally loaded. And so that's where

cumitive layout shift kind of comes

from.

>> One question that comes up to from a lot

of people is this idea of cognitive

surrender.

>> The first is cognitive debt. The erosion

of your ability to have good memory.

Cognitive surrender is where you blindly

give in to whatever the AI says. Its

answer becomes your answer. software

factory meaning

>> instead of focusing [music] on the

prompting, you're building the system

that can do the prompting, but simply

having your loops build everything

without guard rails around the blast

radius is a recipe for disaster.

>> What do you think will be important in

the next couple of years to stay at the

front of the industry? What we are very

likely to see happen next with

engineering careers is

if you've ever opened Chrome dev tools

or optimize a page for core web vitals

you've used software built by Adi

Osmani. [music] Addie spent 14 years at

Google most of it on Chrome going from

software engineer to director of

engineering. And when starting out in

tech as a teenager in rural Ireland he

built his own web browser from the

ground up. Today we talk about the

inside story of Chrome Dev Tools. why it

became the closest thing Google has to

an IDE and the problems that are still

unsolved like memory debugging what

becomes different when you become a

director at Google and how at state

hands-on building software while

building an engineering org with more

than 50 people how AI is changing

software engineering cognitive depth

cognitive surrender loop engineering and

software [music] factories and many more

if you want to hear from someone who has

spent decades helping developers

understand the web and is now thinking

deeply about how AI changes software

engineering this episode is for for you.

This episode is presented by Antithesis.

If you work [music] with agents, your

job is no longer just writing code. It's

specifying and testing it. Antithesis

[music] is the most effective method of

verifying agentic code. Today, before we

get into Add's journey at Google, I

wanted to talk about a really cool

product at Google, Google Cloud Run, and

their recently launched Cloudr Run

Sandboxes. When you're building AI

applications or AI agents, you often

want to run untrusted programs like

execute some Python code the model

generated or run a headless browser to

fetch data from the web or even execute

code submitted by a user. But how do you

make this fast and secure? This is

exactly what Cloudr run sandboxes do.

Cloudr run sandboxes are ephemeral

isolated G Visor environments that spin

up extremely fast. They were built with

security in mind. They enforce

credential and environment isolation.

Basically, they have no access to your

services environment variables or

secrets. Sandboxes also opulate with

lockdown network egress. Deny by default

to the internet. Accelerate development.

Eliminate infrastructure toil and run

untrusted workloads with confidence on

Google Cloud Run. Try cloudr run

sandboxes today at cloud.run.

Addy, it's so nice to have you in person

on the podcast.

>> Oh, thank you for having me. And before

we kick off into your career and how you

got started, I wanted to ask just before

we started recording, we were talking

about how has your day-to-day workflow

changed recently a bunch thanks to all

these tools.

>> So agents have allowed me to take the

improbable and turn it into the possible

in ways that are kind of weird and

wonderful every day. I I I manage a lot

of my life now using agents. And one

example uh just from you know this week

we've got the AI world fair happening in

San Francisco. Um I'm doing a closing

keynote a couple of days in. And so I

wanted to make sure I wasn't repeating

any beats that other speakers had gone

into depth then. Um and I also wanted to

make sure there was good connective

tissue from my talk to a lot of the

other sessions that had happened. Now,

normally in the old days, you kind of

pray and hope that there was any content

from these sessions online and maybe

you'd look through the abstracts. I was

able to fire off a bunch of agents, you

know, go through everything that you can

find about the talks from the last

couple of days, look at the abstracts,

any social content, anything around that

that can be useful. And that was able to

help me kind of sculpt what I already

wanted to talk about into something that

I hope is refined and will give people a

way to connect from other parts of this

conference back to, you know, the way

that I'm going to close it up. That for

me just feels very empowering. You know,

something that would have taken a very

long time if it if I would have even

been able to do it at all is now very

much within reach. And then you

mentioned that it feels like and a lot

of people tell you like there's like

kind of like both fun and chaos at the

same time right now like everywhere,

right?

>> Yeah. Yeah. Absolutely. Fun and chaos. I

think that you know for many of us when

you have a lot of ideas or a lot of

vision you're often bounded by time or

how much can I actually do? And in some

ways agents have unchained us. Um, and

this is one of those reasons why, you

know, you keep hearing, "Oh, hey, what

what are you doing with all of that time

agents have freed up while I'm doing

more work?" I think for many of us, if

we didn't enjoy it, we wouldn't be

filling that time up with work. But

we're having fun with it. And so, it's

fun thriving in that chaos.

>> Yeah. But now, take me back to the very

beginning. A lot of us know you and I've

gotten to know you when through your

work at Google, through your books, but

I'd like to start from even before.

Where did you start out? How did you

have your first contact with computers

and how did you build a web browser in

when you were a teenager in high school?

>> So, I've always been fascinated with

understanding how things work and I grew

up in uh rural Ireland um which was at

times you know we didn't necessarily

have the best internet connectivity.

This was back in the days of Dialup. Uh,

and so my first contact with computers

was, uh, you know, I was probably eight

or nine years old. We were very

fortunate that my dad was able to get

us, uh, our first desktop machine. And,

you know, I'd play around with apps. I'd

play around with just like trying to

browse the internet. And it was always

fascinating to me like, how does any of

this work? I'm just typing in something

into an address bar and all this

information is just rendering somehow.

Um, I'm getting back text, photos,

videos. Like, how does any of this stuff

work? And so over time, like I would

build out websites. I'd start to get

into programming. My very first

programming language was Pascal. Um, so

I'm big big fan of the the Borland tool

suite. Uh I learned C++ uh when I was

fairly young and there was one year when

um I noticed that we had a a kind of

popular national science competition and

traditionally that competition was very

much about you know hey do students have

interesting breakthroughs or thoughts on

physics or chemistry or any of those

things but this the the year that I'm

talking about was the first year where

they actually started to really take

computing seriously and as I mentioned

um I didn't have the best internet

connection. Uh this was also during the

time when especially uh if you were a

teenager, you started to get into, you

know, learning about downloading stuff

>> and we didn't have fast internet

connections back then. If you cared

about, you know, checking out a song,

you could be waiting hours for that to

download. if you cared about trying out

a music video, man, that could be a

night, two days sometimes to download

it. And so I tried to to study how these

kind of download managers that were

popping up worked. And download managers

kind of offered this one hook. Well,

rather than making one connection to a

server, what if we spawned multiple

threads and made multiple connections to

a server and we kind of chunked content?

you know, it's classical computer

science, you know, break down problems

into smaller chunks. And so that was one

of the ways. And if the server

supported, you know, uh, chunking, you

were able to in some cases actually get

your file, uh, downloaded a little bit

faster. And so it dawned on me like,

hey, we're using this technique for

downloading individual files. Has anyone

applied this to how we browse the web

pages?

>> Yeah. And so obviously I can't I

couldn't you know do something

complicated before I you know took my

first baby steps. And so I thought okay

I'm going to try exploring how you build

a browser. I started to read uh you know

specifications. I was probably 15 years

old when I started this 15 16 but I

started to read specifications. Okay

HTML CSS JavaScript. One of the things

that I gained a great deal of respect

for and I still have a lot of respect

for is developers throw all kinds of

weird crap at browsers and yet they

still render something. Right? If you

ever want um you know if you ever want

an interesting uh experiment in in

therapy, if you're ever feeling bad

about your code, open up the dev tools

and just browse the web for 10 minutes.

the number of things that will go wrong

and yet you'll still be able to probably

interact with the site is just wild. And

I had that

>> and going wrong you can just look at the

warnings and errors honestly just that.

>> Yeah. And so that was one of the most

complicated things when I was trying to

build a browser. It was like, yeah, you

can parse HTML, you can parse documents,

you can load up images, but as soon as

you run into pages that stop following

those specs, and they take a very loose

interpretation of what's supported, you

have to really, you know, roll your

sleeves up and try to behave the way

that actual consumer browsers did. And

so I had my fun um building out a

browser. Adding interactivity and

JavaScript support was very difficult.

Uh managed to get it working and then

I'm a sucker for pain because I decided

well I guess uh applets and flash and

you know we had Windows Media Player

back then so people were like embedding

all kinds of interesting content. I told

myself, well, it's not a complete

browser if it doesn't support all these

other things. And so, I added support

for them. And then I could finally get

on to what I actually wanted to work on,

which was exploring if I could speed up

web browsing for this point in time when

you were kind of constrained by the

hardware and uh bandwidth that was

available locally. Back then, this was

this was a personal pain because before

um before I could get a good internet

connection, I would literally every

weekend I'd specially wear cargo pants

that had a lot of pockets and I would

fill my cargo pants with floppy discs

and I would walk down to our local

library that just happened to have a

slightly faster internet connection and

I would try to like save as much as I

could then go back home, check it out on

my computer. And so this was a personal

mission for me. I really really wanted a

faster internet connection. But I

finally got to explore this idea. It

worked and in some you know back then in

many cases there were servers that

supported this idea. Um it did make

things a little bit faster and so I you

know I ended up building something that

worked for me. I uh took it to this

national science competition. Um I was

nobody. I am nobody but I was nobody. I

was just this kid and I was kind of half

expecting to just leave the competition

and go back home at the end of it and

say like, "Yeah, you know, I showed some

people some cool stuff." There was like

a big live audience at the tail end of

this whole event. It was, you know, on

live TV and everything. And when they

called out the overall winner, I was

shocked cuz I did not I didn't expect to

win this thing. And that was my kind of

first taste of media attention. It was

very strange. Um, uh, the weekend right

after, you know, you're a kid, you you

kind of like want to sleep in on a

Sunday morning. You don't really have

too many people back then like calling

your cell phone or whatever.

>> Yeah.

>> Um, and my my cell phone is just like

not stopping ring. The first call is

like from the Wall Street Journal. And

>> basically, you kind of like went viral

in a time where there was not even

Twitter, right? Like there was like none

of this just yet.

>> Yeah. It was like, yeah, it's Wall

Street Journal, CNN. I didn't really

understand what was happening, but it

was it was my first taste of that world.

One of the things that I learned from

that experience was I still didn't fully

understand everything that I was doing.

You know, you're a teenager, just

because you can build an app that runs

on your machine and accomplishes a goal

doesn't mean that you understand all of

those layers behind the scenes. And that

I think kicked off for me a

lifelong thirst for knowledge and

understanding how things work. In in

some ways I can call myself a one-trick

pony. I I care about understanding

problems and how you know how to fix

them, how they work behind the scenes.

Um I would go on to work at startups. I

worked at AOL at one point. Uh again

continuing this theme of of working with

with browsers. When I joined AOL um I

had a very AOL uh moment. my first day

uh my manager was was a very kind guy

said like hey yeah you know go help the

team just log into the browser go help

the team it's like okay cool I'll I'll

do that so I fire up um the AOL browser

app and I have my work machine this app

is loaded up and before I can debug

anything the first thing it asks me for

is my credit card I'm like sorry what I

need to enter in my credit card to even

start my work and my manager is busy so

I don't know if there was like some

workaround. I was like, "Okay, I guess I

guess this is what I need to do to to

get started with work."

>> Wow.

>> Different time. Very different time.

>> But few years later, I would join Google

and I would work on Chrome. But

something for me that's been very

interesting is, you know, if you treat

computing as this onion where you just

keep peeling back the layers, there's

always something interesting behind the

scenes. And the more of those layers

that you can peel back and understand, I

think in some ways the better you can

optimize for that world, you know, there

was a time when I maybe only understood

the surface of how things render. But if

you're talking about browsers, you know,

there is the network, there's

compositing, there's the JavaScript

engine, you go down another layer,

there's chips, there's memory, there's

GPU. And the more you understand about

all of these different foundational

pieces, the better you can then optimize

and build something that you know um can

serve people even on constrained

environments. So if you have a a

slightly slower phone, well, I

understand now why the phone is slower

and what constraints might require us to

think slightly differently about what

we're building there. So I'm always a

big fan of encouraging people to

understand how things work. And then in

the spirit of understanding one project

that you got involved on early on, you

know, you built your custom browser that

did some cool stuff and you understood

how to render, parse, do some of these

things and then you join into the jQuery

project. How did that happen?

>> Yeah. Um,

>> and in jQuery for a while after you

joined it, it it became for a while the

most used library in the JavaScript

ecosystem. So basically across the web

there there were a few years of that.

Yeah, I think full kudos goes to John

Ric, the creator of jQuery. Um, jQuery

was really my first contribution to a

big community open source project and

John was someone that was very welcoming

and created an environment where people

could, you know, learn and become better

open source contributors. Uh I started

off uh working with uh the team of

people that would deal with triage and

issues and then moved on to like working

on blog posts and contributing to code

and other ways. But you know as you said

it it was so widely used that you end up

with so many different use cases people

have. And a lot of the times back then

people would have very strong opinions

about like hey this thing should be in

the main library in core versus being a

plugin. And I got a lot of respect for

how you effectively like work with a

community while also holding a line in

terms of, you know, what decisions

should be made to optimize for long-term

maintainability, for example. But it was

a great experience. Um, and that helped

me kind of carry those lessons on when I

worked on my own open source projects. I

was very grateful for that opportunity.

>> Yeah. And one of your popular open

source projects back in the day was

called Tudu MVC. Yeah. Can we talk about

what it was and why you started?

>> Yeah, absolutely. There was um a point

in time back in the dark ages of

JavaScript when we didn't have

frameworks and we didn't have libraries.

Over time, those things started to pop

up and uh we began to have quite a few

of them and they all tried to accomplish

in some some cases overlapping goals,

sometimes adjacent goals. And we're

talking like Angular. We're talking

about Angular, Backbone,

YUI, X.js. And if you know, if you're

too young for any of these terms to mean

anything, that's also totally okay. Um,

but there were there was this burgeoning

community of libraries and frameworks

that were starting to pop up. And one

thing that I personally struggled with

was, well, how do these things differ?

you know, you can go and you can check

out the landing page for any of these

projects and they all say like, yeah,

we're going to help you build apps, you

know, easier, but um I was very big into

education and trying to understand how

these things worked. So I started off by

creating basically the same application

in every one of these frameworks and

tried to standardize the functionality

so that if you were in the same position

I was and you just wanted to get a sense

of okay well how does the architecture

philosophy change between these things

how does the syntax differ if they're

telling you to build a component or a

piece of UI what is the position they're

taking on it versus somebody else and so

I got a lot of personal value out of the

way that I was building this this thing

up. And so I put it out into the world.

I had no expectations of it being

useful,

>> but but basically it was you implemented

a to-do app or the same todo app with

the different frameworks and you could

kind of compare how they differ.

>> Yeah. Yeah. And the idea was I wanted an

application that was simple enough for

almost anybody to be able to use and

reason about, but it needed to have

enough interactivity and enough

functionality that you could really kind

of stress test at least some of that

functionality a framework offered. In

some cases, you know, that would be

state management or routing or other

things. And so I put this out into the

world. I was kind of shocked at how many

other developers were running into this

exact same challenge. And the project

quickly took off. It started to get a

lot of stars back in the day. It got

thousands and thousands of stars very

quickly. And I didn't quite know what

was happening. Um and before long, I had

people who were working on new

frameworks or new versions of frameworks

reaching out to me um saying like, "Hey,

this is cool. here's my pull request

with my framework. Can can you add it?

Can we work together on standardizing

it? I met some of my first um true open

source friends through this project. Uh

people who are now, you know, very well

established in their own means like

Cinder Sorhus who's written quite a lot

of node modules over time. This idea of

just giving people a simple enough

application

ended up becoming in some ways a

standard for a number of years. I began

to see that, you know, if a framework

was giving people a tutorial about how

to use them, they would actually use a

to-do MVC app as their baseline. It's

been so many years, that was at the

start of my career in many ways. Even

this last year, I still see labs

sometimes like showing off to do MVC

apps when they're trying to test out

features. And the longevity of this

thing has been um very surprising to me.

Another thing that was surprising was at

one point uh when the project was taking

off uh Apple reached out to me. Yeah,

Apple reached out to me uh and

specifically the people who are working

on Safari and WebKit and they said, you

know, hey, we're interested in working

on a browser benchmark

to help browser vendors understand like

are they doing a good job at being

responsive? And responsive here doesn't

mean responsive in the mobile sense, but

responsive in terms of interactivity and

are we responding to clicks and taps

quickly. They reached out to me and they

said, "Hey, would you like to

collaborate with us on this thing?" And

what that turned out to be um was

Speedometer. Speedometer uh over the

years uh has become the primary

responsiveness benchmark, web

application benchmark for all browsers

>> and has continued to be for a very long

time. and browser vendors now

collaborate together on it. They've kept

it um up to date. So as new frameworks,

as new architectural paradigms um have

come out over the years, uh they've kept

updating it and that in many ways is

like carried that the legacy of that

project through to today and I've been

just very happy that it's given people

value of any kind.

>> And you were building stuff on the side.

You were also working at at

consultancies, AOL at at different

startups. How did Google come along? So

Google was uh an interesting one. Um, I

remember one of my first uh longer

periods of time spent in the US was when

I was uh visiting my my wife and her

parents out in the Midwest and uh I was

uh sitting uh I remember uh in in their

room watching TV and there's this

documentary about Google that came on

and they showed like you know engineer

early engineers that have been working

there and why they enjoyed the

environment And I told myself, you know,

I would love to work in a place like

that someday. Um, I continued to put out

free education into the front end world,

JavaScript world, web app world over the

years. And, uh, at some point, I guess

Google noticed that it was useful to

some people. Um, and so they reached out

and wanted to interview me for, um, a

Devril and and builder role. Um, there

were there was a some tooling that they

were trying to build out at the time

that they thought could be a good uh use

of some of my skills, but also some just

general evangelism they wanted to do in

the tech community. Uh, and you know,

the stars aligned just happened to to

work out and I ended up working on the

Chrome team.

>> And then when you joined, can you tell

us a little bit more about when you

joined the Chrome team? What was what

was Chrome like? What kind of work did

you and the team do? Because now Chrome

is synonym for web browser. I know

there's other browsers and every now and

then of course they have some market

share but Chrome has largely won the

market but back then when you joined

this was not the case just yet was it? I

remember back when I joined uh it was it

was a period when we were very excited

about developers bringing their

creativity to the platform. So what can

you do to push on the platform and show

us both what's possible as well as the

gaps so that we can potentially help

fill those gaps and build better APIs.

So, I remember there was this great

Chrome experiment site that we had back

in the day where we would, you know,

sometimes work with studios or work with

developers and just showcase like, hey,

here's here's a cool WebGL example that

maybe you wouldn't have otherwise come

across. And that served as inspiration

for some people to maybe even go and

then learn more about shaders or, you

know, different libraries. It was also a

period of time when I would say

front-end tooling was uh still very much

heavily evolving. you know,

>> we're talking 2012, 2013.

>> Yeah, we're talking 2012, 2013. Uh, this

was at a time prior to uh what what I

would now call meta framework. So, like

Nex.js for example, a meta framework,

you know, didn't exist. So, we we're

going all the way back to a time when we

didn't have the best build tools even

for front end. We didn't have

>> we didn't have things like ES.

>> Yeah. we didn't have um we didn't

necessarily have well standardized

JavaScript modules you know in all

browsers people were still using you

know um AMD and UMD CommonJS things like

that and uh you know the build tooling

and the scaffolding tooling was still

very much evolving and so this was the

period of time when you went through

things like Grunt for for anyone that

you know maybe we're dating ourselves

but Grunt as a as a built system

>> and also when you debug the browser, you

would use Firebug. You would open it in

Firefox and then hope that like in IE it

would work, but if it didn't, there

weren't many good debugging tools in IE

specifically. Later, they became better,

but back there was a time where there

was not.

>> Yeah. I And I think that, you know, back

back in the heyday, there was a lot of

workarounds people were trying to apply

um to still have a toolbox of some sort

before things got much better. We put uh

some some work into working with um you

know the folks who were building out

build tools and testr runners and

scaffolding tools. We worked on our our

own contribution uh called Yommen back

in the day. And Yman was really about um

I don't know that I'd call it you know

the first meta framework but I would

call it an attempt at trying to bring

just a little bit of organization

uh to your starting point. Yomen was a

scaffolding tool we created where you

would get a wizard in your CLI and you'd

kind of say, well, yeah, I'm trying to

build this thing and maybe I'm

interested in using uh this UI library

and this testing library and uh maybe

I'm interested in deploying to this

target. Now, for folks who are listening

in, those ideas might now sound very

standard and things that you will find

in all the tools you're regularly using.

Back then, they didn't exist. And I

wouldn't be surprised if many of the

modules we created back then are still

being used under the hood for some of

your favorite tools. So it was it was

very fun getting to be a part of that

moment where we were trying to like

figure things out and reduce friction.

But I will say that you know there was

this long period where we kept changing

tools what felt like every every once in

a while, right? you went from Grunt to

Gulp to Webpack to you know to to V

rollup all these all these things kept

evolving and I was happy to see the

evolution but I'm also happy that things

in some ways feel like they've

stabilized.

>> Yeah, there there was I think it's it

was turn but I mean that's when

innovation happens. Did you work on

Google Chrome dev tools?

>> Yeah.

>> How did that start? Because I I remember

in 2012 I'm not sure if there was dev

tools but again there was the

state-of-the-art was Firebug. It wasn't

I think it was open source. It it was

actually just superior debugging on the

web to anything before and I'm not sure

at what point but I do remember you know

Chrome Dev Tools slowly started to

emerge and it it started to bring a

bunch of new stuff like you could do

performance monitoring some of those

things. Can can you tell me from the

inside how how did it start?

>> What what you built how you figured out

what to build? Yeah. So I have to give a

shout out to um Pavle Feldman who was

the tech lead uh for Chrome DevTools and

and really played a very large role in

helping it um come to be originally.

There was this uh period of time when

you know uh Chrome was trying to figure

out how it differentiated its developer

tooling story from WebKit where we had

the you know Safari inspector the WebKit

inspector and I think there's a very

specific direction that was developer

centric um and cared about the ecosystem

that Pavle and his team uh were trying

to to help out with and I noticed that

they had a very good relationship

talking to not just developer

evangelist. So this was the time when we

had really sharp minds like Paul Irish

around also like working very heavily

with the creme dev tools team. We would

later have um folks like Paul Bouse who

uh is now known for things like

impeccable um the impeccable skill for

design and uh I feel like uh one of the

nice things about that period of time

was you had these people who were web

developer archetypes and were builders

on the side myself Paul the Paul's and

we would try to bring those insights to

the dev tools team and help them

understand well here are the areas of

friction that we're running into. In

some cases, you can't just build tools

to help you out with them because you

don't have the underlying

instrumentation.

Um, and so I was very happy to see

things like uh performance tooling

heavily evolve over the years. Like the

DevTools performance panel is just an

amazing piece of technology. The fact

that you can just hit record, start

interacting with your page, and you get

a flame graph, you get very deep tracing

about where all the time is being spent.

And that continued to evolve over time.

And then we had, you know, really hard

problems. Uh, you know, some of the

hardest problems have been around

memory, right? I would say sometimes, I

don't know if it's controversial, that

very few developers understand memory

management, and that makes it even

harder to debug memory problems. And so,

the state-of-the-art around memory

debugging hasn't evolved all that much

over the years, but it's a hard problem.

uh the dev tools team tackled a lot of

interesting hard problems. Can can we

talk about a part that where you brought

in something new cuz you know debugging

memory just back in the day it's pretty

much I mean if you have a language that

has let's say heap you can try to

visualize what's on there

>> you can attempt and maybe be successful

at allocating which variables there are

and then you can try to also I mean some

variables are are the easy part there's

also stacks and you know it gets a

little bit messy but you you you

basically have a memory and and you're

typically interested in what is growing

and there's a part that I I don't I

don't I don't know that we got too far

on that but you're kind of trying to see

is this is this getting bigger? What are

the loops? Where's my stack?

>> Absolutely. I would say that there are a

few interesting arcs where we were

seeing you know ourselves and developers

externally running into certain kinds of

friction and um you know work with the

dev tools team to try evolving some

tooling in that direction. One of the

the big arcs was embracing the fact that

developers were increasingly using

frameworks and libraries to build for

the web. Y

>> now um for anyone that remembers those

dark ages, imagine that you have a page

that's very interactive. It's using lots

of different libraries and you're trying

to debug what's happened. What part of

that code do you actually care about? Do

you care about the framework code that

is powering things behind the scenes? Do

you care about the plugins or the

components sitting on top of it that you

haven't written? Do you care about the

code you yourself have written? And so

you have all of these very nuanced

aspects of debugging that need a

solution. One of the things that we we

tried to introduce was just this respect

and understanding that yeah developers

are going to be using these different

tech stacks. You know, we had a source

maps story sitting there. Yeah. Where

potentially we can start to reason about

what's in

>> you map back to like what part of the

code

>> exactly

>> which is not trivial.

>> Exactly. Which is not trivial. And a lot

of kudos to the team because I think we

ended up on a source map story that

really helps you reason well about you

know even if you're using a long tool

chain of things like if you take a look

at any tools that developers for any big

site you know whether it's Uber or

Netflix or any large Twitter any large

site you probably underestimate the

complexity and the number of tools that

you're running at any one time for any

one task you know and being able to

still allow people to see well hey

here's actually the files that you care

about. It's a hard problem. I think that

allowing people to get that view was

part of the value that we we brought. We

introduced different kinds of blackbox

uh views over the years so that you

could say, well, hey, actually I know

that I don't care about you telling me

there's an issue with, for example, the

React library, but I do want you to tell

me that there's an issue with the React

code that I wrote. And so giving you

even those toggles, those controls I

think was very powerful for people.

Mobile was another big moment that

changed everything. Um, and you know, if

you think about mobile, um, today I

would say, you know, there's probably

established best practices around the

things to test, right? Like you want to

test out your viewport width, your tap

targets, like is, you know, if I'm

tapping on something exactly, is it big

enough?

>> Exactly. you know, there are all these

different kinds of sensors even that

mobile devices have. We didn't we didn't

have tooling around any of this stuff

originally. And so we ended up building

out a nice device mode in dev tools that

would allow you to preview, you know,

what your site would look like at

different um viewport sizes. You can

very quickly kind of toggle and say,

"Yeah, this is what it roughly looks

like on an iPhone or a Pixel device."

And of course, you know, um the the

absolute best kind of testing would be

trying it out on one of those app does

accurate devices, but even to quickly

get a sense of whether you're heading in

the right direction was was very

valuable to people. And we would evolve

that over time as more of those best

practices started to establish. I guess

the web apps growing up um so PWA,

progressive web apps.

>> Yeah. Uh there was a period of time when

you know people really wanted to make

the web uh competitive compared to

native. And so you think about well what

are the things that are missing? Well uh

you need a really good story for offline

caching push notifications background

sync all of these capabilities that you

know we didn't necessarily have a strong

story for. And because these are

non-trivial features, you need to have a

debugging story around all of them. And

so we help build out the application

panel so that you can go in and for any

of these features, whether it's

debugging service workers or it's

debugging your cache or debugging any of

these things, you're able to do that.

And so even though the tool set has

expanded over time for each of these

eras, I feel like DevTools has been able

to keep up, especially as the APIs in

the browser has also been evolving over

time to meet these moments. Well, that's

interesting because I I usually when I

look through different companies and

their strengths, Microsoft is amazing at

building ideides and so is for example

Jet Brains, but for Google, I never felt

that Google was any good at building

except for inside of Chrome. like

whenever I I have to debug a web

application, I always I the past like

many many years I use Chrome DevTools

because it it had I mean the kind of

debug functionality I'm used to having

Visual Studio have which is breakpoints,

conditional break points, all sorts of

so many debug options from as as you we

just said performance memory being able

to simulate some of those things. So

it's very interesting to for me to see

that it's almost as if I'm not sure if

this was you, your team or Google as a

whole, but they realized browser is very

important. And so so they built like

almost like an it's almost like an IDE

inside of it. You can you can edit the

things in line and I think as engineers

or as developers unless you work in

front end you never really notice this

but when you do it's fascinating how how

how it came together.

>> Yeah, it's really fascinating and I

think that uh are we an ID? Aren't we an

in ID? Is that a direction we want to go

in? Was always a hot topic uh for the

team. And I think that where things kind

of landed was, well, we want to meet

developers where they're at because

you're always going to have your

favorite, you know, editor. Now we're

talking about, you know, your your

control planes for your agents. You're

always going to have a different

surface, right, that you want to

primarily work in. And as long as

DevTools can meet you where you're at

and be useful, I think that that that's

been something the team has tried to do.

We continued having other eras. Um, Yong

Gao uh became uh our next tech lead uh

after after Pavle and helped us through

the era of trying to figure out AI is

now in the picture and we want to both

be able to help humans reason through

this massive amount of data that the

browser generates for you as well as

make it possible for you to connect your

agent up to Chrome and DevTools and be

able to have it just automate, you know,

a lot of these journeys for you. And so

I think that for the first of those

problems, uh, I remember anytime I would

work with a big site on their

performance problems, uh, you could

easily spend half a day, um, you know,

just looking at traces before you've

even written any fixes at all. And now

that we have LLMs, it's very quick to

like reason through massive stack traces

and actually be able to get down to

fixes you can make. And that's just been

really, really wonderful to see happen.

Addy just described using LMS to go from

massive tax issues to working fixes,

which is the perfect moment to talk

about our season sponsor, Sentry. You

probably already know what Sentry is

because you're a developer. If not, just

ask a dev and they'll tell you. I use

Sentry to monitor the back end of the

pragmatic entry for any and all errors.

Of course, Sentry doesn't only do

errors. They also have logs, replay,

spans, profiles, metrics, and more

because they're all connected by the

same trace. One new capability Sentry

has built that I'm really liking is the

ability to fix errors. Let me show you.

Here's the list of errors on my admin

back end. There's a recent error on O

that I want to check out. Let's have

Seir run an autofix for us. Seir is

Century's AI debugging tool. First, it

generates a root cause analysis. It's

finding some problem with HTTP versus

HTTPS URLs. Cool. Now that we know

what's going wrong, Seir can create a

plan on how to go about fixing it. I

could go and edit this plan, but I'm

happy with it. So, let's create an

actual code fix. Here's a code fix that

Seir generated. Assuming it looks good,

and in my case it does, let's draft a

pull request. And boom, the pull request

is created, ready to merge. What I love

about autofix is how Centry went from

showing a list of errors inside my

application to offering me a fast way to

fix it and close the loop while I stay

in charge of this bug fix the whole

time. Debugging just got a whole lot

faster and a whole lot easier. Check out

Sentry at centry.io/pragmatic

and start detecting errors, diagnosing

your root causes, and fixing issues and

regressions today. Addie mentioned

things that change when we work with

LMS. One thing is for sure, if you work

with agents, your job is no longer

writing code. It's specifying and

testing it. And this leads us to our

presenting sponsor, Antithesis. Anthesis

is the most effective method of

verifying agentic code today. Let me

explain how it works. Antithesis runs

your whole system in a hostile

simulation. By doing so, it finds every

bug before your users do. And because

the simulation is fully deterministic,

anticys doesn't only find bugs. It gives

you a perfect reproduction of every

issue. To create such a tool, the

antithesis team needed to invent new

kinds of debugging tools as well. For

example, here's what's called a buck

probability graph. The x-axis is virtual

time and the y-axis is probability. As

anticis runs the hostile simulations, it

plots time frames when the bug

probability increases, which greatly

helps with finding the root cause of

bugs. And anticysis also has a log

visualizer. Vertical lines going down

represent events branching off from the

same state. And the purple dots are

where the buck happens. Antithesis is as

good as it gets being able to ship agent

written code. It's what teams at Jane

Street, fly.io, and the Etcdity

community use to ship with full

confidence. Head to

antithesis.com/pragmatic

to learn more. And with this, let's get

back to Atti and talk about core web

vitals. And one area that you really

pushed you and the team, pushed the

industry together is core web vitals.

You know these are a standardized set of

metrics to just figure out the real

world experience of web pages and some

of the you know before this again you

would as a developer you would measure

like all right how quick does it render

or how quick does it download it was

very simple stuff but you introduced

things like LCP largest contentful paint

cls commumulative layout shift fie first

input delay and then in impaction to

next paint like you were there like how

did the team come up with these things

there. If you're not a web engineer,

there's still it takes a little time to

understand them. But but it does

actually explain like how users feel

like I feel you somehow inside of Google

managed to connect the kind of feel to a

number. I think that um the Chrome team

uh has always had an appreciation for

user experience research and um again

every every time there was uh a new

moment for the web would reconsult

well what are users expectations and how

can we help meet them. The way that we

used to reason about performance was

very much like hey is a page loading and

what does that even mean? Well, for many

people, is the page ready? But what does

ready mean? Does that mean that I see

it? Does it mean that I can click around

it and anything actually happens? Um,

and so I think for a very long time, we

had this this almost nebulous way of

thinking about page load times. And the

team felt like it was finally time to

come up with a more nuanced perspective

around how we reason about performance.

And so if you break it down, there are a

number of key moments across the user's

journey that they care about. Is it

happening? Is anything loading? You

know, do you see a header? Do you see a

spinner? Do you see anything at all? Is

there something useful there for you? So

>> maybe that's a header image. Maybe it's

a hero image. Maybe it is a hero video.

Maybe it's like the core piece of

content on the page. Is it useful? Is it

usable right there? And all of these

different moments can correlate to these

different metrics. So for things like

your hero image, you can think about

that as your largest contentful paint.

And that's not going to generalize

across every page. In some cases, the

image may not be the most important

thing. It might be, you know, the

article text. There may be cases where

uh you know, you want to be able to

interact fairly quickly with a page. I

can remember um many times over the

years when I might be shopping and uh

whether it's on my phone or on my

desktop, I will click like the add to

cart button and just crickets.

>> Yeah.

>> Nothing will happen

>> cuz the JavaScript did not load or the

event hand or or the or maybe the event

handler was not attached because not all

elements finished loading. We know as

engineers what's happening but as a user

it's like

>> yeah as a user like wait what's what's

happening

>> and then stuff can happen where you you

just tap tap tap the event handler gets

attached and now you're adding it like

twice or three times but you don't know

and yeah

>> humans are humans are are shockingly

simple. You know uh if you think about

the experience you have with somebody

that's just trying to cross the street.

If uh you know if the light doesn't turn

you know it doesn't say they can walk

fast enough they'll just keep hitting

that button. That's the same experience

they have on the internet. I think that

there were other aspects of user

experience that um I think we

acknowledged were actually kind of

problematic. One big one was over the

years obviously sites tried to monetize

as heavily as they could. And so you

would see not just banner ads but you'd

see modals, you'd see all of these

things thrown up in front of your face.

And you know, even if you set aside, you

know, maybe there's some validity around

a business needs to monetize, those

things shouldn't cause a really bad

experience. If I'm on a page that has a

bunch of ads, I shouldn't start reading

an article and then suddenly everything

gets pushed down,

>> right? Just because the ad is finally

loaded. And so that's where cumive

layout shift kind of uh comes from. It's

this idea that, hey, we should be trying

to keep that page stable so the user has

a good time. There was a lot of

iteration around how do we define these

metrics in a way that captures a few of

these different use cases that are very

nuanced because the internet is not all

that homogeneous. There are lots of

different ways that a person can think

about the value of a page and what's

important. And so the team did a lot of

experiments, experimented with lots of

different ways of thinking about these

metrics and worked very heavily with

both the standards community and

developers to validate like hey do you

actually believe that these things line

up with how you would say you think

about the value of your pages. One of

the things that I always find

fascinating is um there are some

companies where you know they they have

thought from the ground up like if you

start from a blank white screen what is

actually important to the user end to

end and then there are many companies

where they haven't for whatever reason

time or they just didn't think about it

they haven't gone through that journey

and so core vitals allowed them to

finally get a more nuanced conversation

going about like hey what's actually

important to us how can we make sure

that whatever key action the user has to

take, they can do it pretty quickly and

be guaranteed that they're not going to

have a bad time.

>> Now, you spent 14 years inside of

Google. You most of it was inside a

Chrome organization. Later, you moved

over to to cloud AI and work with Genai

as well. We've done research before on

Google's engineering culture, but can

you can you summarize like what it felt

working there in terms of what and

especially comparing to the startups

that you worked before? You also talk

with with companies now outside of

Google. what were things that were

uniquely Google?

>> As part of my Google journey, there was

a lot of work that I did on the

developer side. There's also a lot of

work that I did on the consumer side. So

working on um Chrome performance for

example and when you're working on

something that goes out to billions and

billions of users which is the case for

for many Google products now um the way

that you think about engineering culture

velocity experimentation is very very

different I think than sometimes how a

startup might approach things especially

if you're trying to move very fast when

we try to make a change inside a browser

that has a global audience with a lot of

people. There's a lot of experimentation

that has to happen and a lot of

experimentation that also requires just

like testing out well hey does this

problem not have one solution but

actually a couple of different ones

depending on what market you're in and

how do you evaluate success when maybe

we have 20 other experiments or hundred

other experiments happening at the same

time and so the AB testing culture I

would say was a very big thing and could

to the Chrome team for having what is

now I would say a fairly um stable and

rigorous process for being able to try

those things out in the real world. I

also felt that even though sometimes

from the outside it didn't necessarily

always uh come across zoomed out at the

Google level, I did feel like there were

many people that cared a lot about

developer goodwill and developer

sentiment. But when you have, you know,

a very large company, obviously you're

not, you know, you're not, it's going to

be very challenging to have every group

talking to every other group.

>> Impossible. We always made best efforts

to try, you know, um, getting to a place

where we were doing the right things for

developers as best we could, but I was

glad to see that sentiment and that

level of care for the community and for

our users. I also appreciated that

Google was um, was open to change. So, I

would say if if I had to summarize my my

one big change contribution to Chrome's

culture, it would be meeting developers

where they're at. and that embrace of

people are going to use whatever tech

they want to use. You can't tell people

what to use very often. They're going to

use whatever they want and your job is

to help them be successful on your

platform and to help their users have a

great time. I think that there were a

lot of decisions we made over the years

that helped make that um a little bit

more possible and uh you know we had

good collaborations with different

framework teams. we took their feedback

about APIs they would like to see in the

platform. It became a lot more of a

collaboration with the community rather

than kind of guessing what we thought,

you know, the community needed to be

successful. And so I was very happy to

see that happen. I would also say that

>> Google was very good at allowing um

different parts of the company to share

their learnings towards some point of

convergence. Uh so for example the

software engineering at Google book one

of my favorite books.

>> I was very happy to see different

flavors of that over the years

internally at the company because you

work at such a big company. Um you're

always curious well what is best

practice right like is there a best

practice?

>> So so there were like internal writings

of like here's how this org is doing

some parts of software engineering or

building or experimentation or whatever.

>> Yeah. One of my favorite things to do

was um you know I was curious well my

team might have a perspective on testing

or user experience but how does the

YouTube team think about it and are

there parallels are there things that we

could learn from each other and there

certainly were you know even looking at

how other people think about the world

can sometimes lead to collaboration

opportunities. Um we actually worked

with uh the YouTube team um to improve

their core app vitals at one point you

know and they were they were excited to

see that there were just more refined

metrics and ways of thinking about

experience.

>> I mean I mean I guess it's just

important to point out that this

collaborative nature is not a given in

any all large companies. There are some

companies don't want to name names right

now but where organizations

don't feel that they're incentivized to

work with each other because they might

have different goals and it's not that

they hate each other. just like focus on

themselves and it can feel a lot more I

guess political in that sense. Yeah, we

talk a lot about high agency these days

and um I think that sometimes when you

see those collaborations happen, it's

because there are people with enough

agency on both sides that they want to

make it happen. Um and they see the

mutual value in collaborating because

exploring, you know, how to improve the

user experience for something like

YouTube, it was extremely nuanced,

extremely educational, very nuanced, but

also took a very long time. and we just

felt like the value was there. I was

glad that we could make it happen.

>> Yeah.

>> Can we talk about your specific career

path inside of Google? So, you spent 14

years there, which is a very long tenure

and I'm starting to develop a bias for

like it's nice to have long tenure

somewhere at some point in your career.

There's a lot of values. You were just

talking with uh with with Simon uh the

founder of Turbo Buffer about this

earlier. What level did you get in? How

was your career progression? At what

point did you become a manager? And how

did you think about things like career

compensation

growing?

>> Yeah. Um, so I started my Google career

uh back when I was living in the UK. Um,

actually,

>> so you joined Google UK. Yeah, I joined

Google UK originally and uh I believe I

joined at a level four like at the

>> Yeah, that was one the mid-level

software engineer

>> um back then and I was a developer

relations engineer. So a person that's

in Devril but you're a little bit more

focused on you know the builder side of

things. Over the years I kind of got

promoted in that role to like uh level

five and level six. I became a manager

within Devril and

>> when you were at level six at the staff

level.

>> Yeah. Yeah. And then I um was was

leading uh part of the Devril team uh

and at some point maybe five or six

years in um I started to feel like you

know I I loved doing developer relations

but I I I am very much a builder at

heart. I love I love engineering and I

love product. I love all of it you know

but

>> I get it. I was very curious, you know,

um what it would be like to be on the

other side of that because I'd been in

engineering prior to Google. I hadn't

been, you know, in an official Devril

position prior to that. And I was

interested in going back down that

direction. And so over the years, I

transitioned uh back into kind of

software engineering and specifically

like an engineering manager role. um

that gave me the flexibility to both do

like engineering work but also manage uh

teams.

>> But you just had a smaller team at that

point.

>> At the at the start um had a smaller

team and then it grew out. Uh I would

say the average at one point was

probably in the 45s to 50s. I I think

that depending on where you are in your

leadership or manager journey, you know,

success means different things. not not

success from a career perspective but

just success for the organization

because ultimately what you want to get

to is a place where ideally the team is

almost self-sufficient and uh I write

about this a little bit in um my book

leading effective engineering teams but

you want to get to a point where um you

know your your machine your org is

self-sufficient enough that you know you

just need to occasionally tap the blimp

make sure that things are working you

can course correct if it's Not, but that

frees you up to then focus on the next

important sets of problems that the org

needs to, you know, tackle heads on. And

that allowed me, for example, to um

really get deep into thinking about,

okay, well, model quality is starting to

get better. What does that mean for

developers? What does that mean for

developer tooling? What does it mean for

how we think about benchmarks and

collaborations with third party vendors

and all of these other things that are

part of developer success? And so I was

I was glad that I had that time and then

I could take those learnings back to the

team and work with them to evolve us

into this moment where we could you know

help developers maximize uh how useful

dev tools can be and dev tools and

chrome can be for agents.

>> So do I understand correctly that you

know you you were an indiv individual

contributor you were going up the career

ladder which is somewhat expected at a

at a large company like Google with the

right mentors and the right structure.

And then when you became a manager and

you switched but you decided to build a

bit more you then focused on you still

had a growing and increasingly large

team and 40 people that's that's not a

small team but you you tried to help the

team fix any issues help them mostly run

by themselves so that you would have

some time to actually do some individual

contributor like work so you can keep

your hands dirty but also help the team.

>> Yeah. So like it it seems like you do I

understand that you just prioritize to

have that time to build because of

course when you're a manager this could

easily suck up all of your time.

>> Oh yeah, absolutely. And I don't want to

make small of of all the work it takes

to get to that point because a lot of

management is trying to work towards

that point. You need to build out a team

structure like

>> and in some cases you have managers

managing

>> managers

>> managers right of other teams and we had

a global team of people of course that

comes with navigating time zones and

coordination overhead and communication

and all those things and so I think that

we were we were fortunate that we were

able to get to a place where the team

was um largely pretty effective and we

were able to create more of the space

and then I take those learnings and I

try to help some of my other managers

like how do you how do you create this

space now for you so that you can also

help us on this journey of modernizing

for the AI moment. Um so I went I went

from L6 to L7 um to director um in my

>> is L8 or L78. Director's L8.

>> Oh wow. So that that's kind of Well,

congrats. It's it's it becomes every

level becomes somewhat harder and harder

as I understand. But did you care too

much about the actual levels or was it

more about the work and things just

followed?

>> I think for a very long time um it was

about the work but also like as you as

you get to a point where you feel like

the organization is in a healthy place

um you do start thinking about okay well

next level of my career taking on

different kinds of problems like the

next challenge right? the next

challenge. And so I was very much um

wanting to go for a director kind of

promotion for for quite a while and I

was working towards that. And I think

that you know for anyone that's gone

through career changes or promotions,

you know, you know that you kind of have

to be doing the job for a while before

you get it. And

>> um what kind of got you where you are is

when it's what's going to get you to

that next level, right? It's a different

set of challenges. And so I was excited

to, you know, get to start experiencing

those kinds of challenges and and

working more across Google, you know,

working more with our VP and SVP layers

to try figuring out, well, yeah, what

what does the next couple of years or

what does the next year look like for

Android, for Chrome, for our different

platform teams as we're going through

these kind of revolutionary moments?

We're trying to rethink everything. I

did want to ask because we have a lot of

pretty experienced viewers and and

listeners what is the difference in

becoming a director at Google

specifically because director that's the

first executive level I mean different

companies call different but like it's

the first one which it might be included

in terms of responsibility

uh weight on your shoulder because it

does feel like that feels in the

management chain that is the biggest

jump at a large company like this

>> I think that a good way to think about

it is when I was coming up through the

ranks, uh, your director was very often

your first point of contact, as you

said, at the executive level. They would

be the ones who would be keeping you on

the hook for making sure that any of

your annual goals, quarterly goals, any

of that was on track.

>> They'd be the ones that you'd be looking

to sponsor any large programs, any new

projects, things like that. if things

were going like way off track and you

were, you know, being held accountable,

the directors were often the ones that

would be having review forums regularly

to make sure that that whole ship is

actually still steering in the right

direction. And so there is an increased

feeling of accountability at that level.

Um, you have to pay attention to the

details. I think that you can't be

successful in that role if you're kind

of just letting go. And when I say like

you you want ideally to have a self

running org, it's not about letting go

entirely at all, but it's about having

enough of a system in place where you

get the information you need. Any

decisions, any blocks that your teams

are running into are surfaced quickly to

you so you can help them unblock them. I

think that that's really one of the

biggest pieces like making sure that the

business goals get done and making sure

that people who perhaps sometimes don't

necessarily understand how to connect

the tech uh that's being done back to

the business goals like see that through

line very clearly. I remember that um

you know and I was doing the director

role through my time working on Gemini

and cloud AI. I was responsible for some

of our like one of our top goals for the

year. You're expected to report on that

every week or two and be held

accountable. So you need to do do to

make sure that everything happens to

keep those numbers and those goals

moving in the right direction. So

there's a lot of accountability I would

say that comes with. So, so it sounds

like it's almost like if you're juggling

stuff, you're given like two extra

balls, which is like now you both the

accountability, communicating upwards

with with exing the business goals while

doing everything else in terms of like

running now probably larger team being

able to deep dive into the details. So,

keeping yourself up to date. So, yeah.

Well, I guess it kind of makes sense

that there's a trajectory where if the

longer you work in an organization, the

more context you'll have, the more ready

you often become.

>> Absolutely. Absolutely. And and I think

an interesting um anecdote that I think

is worth sharing is and I don't think

this was specific to Google. One of the

things I found most exciting in the last

couple of years was seeing as model

quality's gotten better and and

harnesses and tools have gotten better,

how many people um that were directors

or VPs or SVPs or any of these levels

were actually rolling up their sleeves

and trying things out. Um, and that was

that was awesome to see because every

week you could then have conversations

with people like, "Hey, what did you

build at the weekend? What models are

you trying out? Like what are you

running into friction with? What

workflows are you using?" And that's not

something that was happening before.

Exacts were very typically, you know,

focused on big company problems or big

work problems. Yeah, but that's been

changing in the last couple of years

>> which lead us very nicely into the next

topic which is how AI in your

observation and experience is changing

software engineering right before we

start talking about one question that

comes up to from a lot of people and

you're also thinking about is this idea

of like cognitive surrender.

>> Yeah,

>> let's get into that.

>> Yeah. So there's two pieces here. Um the

first is cognitive debt. So the more

that you use AI, it's it's sort of the

erosion of your ability to have good

memory and have good understanding of

the problems that you're working on. And

the natural followup to that is

cognitive surrender, which is where you,

you know, you blindly give in to

whatever the AI says as your answer. Its

answer becomes your answer. And so you

start to really let go of critical

thinking and your ability to solve

problems just goes to the wayside. Um, I

think that that's something we want to

avoid because, you know, I'm I'm

personally I'm a I'm a big fan of the

evolution curve we're seeing with

harness engineering and loop engineering

and software factories and all of these

things. I'm very excited about them. Um,

but at the same time, I think that we

still need to understand enough about

how things work so that if something

does go wrong, we're actually able to

fix it and not just hope and pray that

the agent is able to figure things out.

And it was about a year ago where you

wrote about the importance of when

you're working with an agent and this

was before they were as capable as

today. But when you're working with an

agent, read through read through what it

it has, think through and then you know

when it generates the code, read through

that code, make sure you understand. So

like you're kind of like doing a review.

Now we have a lot more powerful agents.

Um we can now some people work like

multiple agents. What is your thinking

on the kind of the reading the going at

the same pace of the agent and and you

know like because there's this friction

of like it's now so much easier to like

not not let go because you want to let

go and have cognitive depth.

[clears throat] It's just like they're

faster and it's pretty good for the most

part

>> with many challenges people run into in

life. Um there's a lack of

intentionality around wanting to avoid

them. And that this is one of those

places where I see this happen quite a

lot. Um, and my thinking on this has

changed a little bit. A year ago, you

know, maybe you would have, you know,

like one thinking message from the agent

saying, "Hey, I'm I'm thinking in the

background." And you'd expand it and you

would see a trajectory and you'd see the

summary of like all the things that are

happening

>> and it was like in speed where you could

follow as well. It's like it's like

every few seconds something coming.

>> Yeah. And now if you're using cloud code

or codeex um it's very possible that 20

or 30 sub agents have fired. I am not

going to click through 30 of those

things to read through their

trajectories. But I do make sure that I

do two things. The first thing I do is I

try to make sure that if there is a

summary at the very end, here are all

the decisions that were made. I will

read through that end to end. If there

hasn't been, I will prompt for that

decision process. And you have to be

careful because you don't want a model

to kind of BS you about like the

decisions that were made because

sometimes it can just like make things

up, right?

>> But It run and and as we know it's not

deliberate necessarily but it runs out

of context window like there's

limitations to these things

>> exactly and there and that goes on to

the second thing I'm a really big fan of

this idea of mutual amplification

if you are working with an agent a

coding agent there are a lot of things

that you can do to make sure that the

agent is getting better every day and

you as an engineer are getting better

every day there are simple things that

can play into that things like even

within in this session or within this

project. Can you log your learnings from

the session? Can you log any decisions

that were made, any friction that you

ran into? Anything that you think is

unique about how you've approached this

problem that I should just keep in mind.

That's intentionality. That's like I I

want to understand how things work

behind the scenes. And as long as you

have that curiosity and that thirst for

at least being just a little bit

curious, I think that you know you can

work with models in a way where you're

still preserving a little bit of your

cognitive understanding about how things

work.

>> One new building block that's coming up

in genic engineering is this idea of

loop engineering and Peter Stainberger

wrote about it, Boris Churnney wrote

about it about running loops. A lot of

us are trying to figure out what loops

exactly are. You also wrote a post about

loops.

What do you think loops are? Or how

should we think of them or is is just

some something that is useful for a few

people? Where are you at with that? a

good way to think about so loops are

part of this journey we are on to

effectively create software factories or

you know some some people like

>> software factory meaning like a thing

where you know like it you give some

instructions you're like you in a

factory like I would like to produce a

car and then there's a fully automated

factory and the car comes out

>> so instead of focusing um on purely the

prompting towards getting an outcome.

You're building the system that can do

the prompting and generate the outcome,

[clears throat]

>> do the testing and verification for you.

And it's effectively

the next step of, you know, every phase

of of software evolution is just like a

rising tide of abstractions. This is the

next abstraction. And it comes with a

lot of nuance because I think that, you

know, if you tell someone, "Yeah, create

a create a system that will just do all

of your work for you." Anyone that's

been in the industry for a while are

going to have obvious questions like,

"What about quality? What are you

actually how are you making sure things

aren't going off the rails?"

>> And so, I think that you have to be very

intentional with, okay, well, what are

the parts of this where you're keeping

the human in the loop? Are you having

your system flag to you that, hey, there

are changes that were touched that

actually, you know, are hitting a pretty

critical part of the system? And you

probably do want human review on this.

But simply just having your loops build

everything without having some guard

rails around the blast radius, without

having guardrails around how you think

about quality, I think is a recipe for

disaster.

>> I hear the analogy of self fracture a

lot of places and again it and of course

dark factory as well. dark factory,

meaning it's a fully owned factories.

Lights are turned off because the robots

don't need to see and you save energy

and and money. But one thing that I I

keep thinking that is off on this

analogy is like, okay, in a factory, you

produce a thing. It could be a car, it

could be a screw, it could be something.

It's it's there and it's done. But with

software, specifically SAS and and most

software that we do, it's not done. We

like when it's finished, we release it

to production and that's where it

crashes, the bugs come out. So I wonder

if this this whole idea of like okay

we'll have a factory that produces the

software and it does all the testing if

in production it's not connected to how

it's running and having that feedback

that's you see what I mean like it's a

different type of factory that we're

talking about. So you hit you hit the

nail exactly on sort of the next phase

of that. You can if you can have a

system that can sort of decide uh what

needs to get built, how to verify, how

to test and all of those things,

>> there's nothing stopping you from then

connecting that up to your telemetry, up

to your other systems, up to user

feedback, up to any other signals that

can help build out the product. You can

connect it up to the product backlog and

you can potentially see a world where

you then have this system that has

access to all of these different signals

for how the product can be improved to a

point where maybe it even could get

proactive.

>> Yeah. And we're seeing there's so many

examples that you can plug it up. For

example, if you're using Sentry, uh,

Sentry has automations where like if an

error fires in Sentry that is net new,

you could have a hook that kicks off

your favorite coding agent where it

one-shots a fix and it puts you in your

review. Now, of course, you took it you

could take it further and you could

allow it to automatically do it, which

sounds like a bad idea today, but you

could do it. And I wonder is when we're

talking about loops, is this, for

example, a loop that we say? And maybe

the the loop is just not a good word for

it. Maybe it's I think I heard workflow.

I heard like or if I say feedback loop.

Okay. Like that might be a better word.

Maybe is it just a wording thing where

like we're a little bit confused with

the

>> Yeah. I mean I I think that given how

fast things are moving, we are very

likely to see new terminology sprout out

every month and some of them will be

good fits and some of them will continue

to require some refinement. So I could

totally see workflow being a better fit

um than loop. But from a visual

perspective, I do I personally do see it

as a loop. Workflow also also works.

>> But but then can you give me examples of

loops that you've you've used or you

have seen people on your team or people

on the university use?

>> Yeah. So um I was just mentioning being

able to connect multiple signals up to

you know your software factory from

production

>> from production

>> and may that be logs or errors or all

those things. So, um I have one app

where um I allow people to submit issues

uh to it if they run into any problems.

And historically, yeah, like a bug

report. And historically, I would, you

know, manually go through everyone and

whenever I had time and then make a call

in terms of like, okay, well, I only

have time to address so and so and so.

Um I can't go through the full backlog.

You can now connect up so many other

sources of data. You can connect up your

Google Analytics. you can connect up,

you know, if you're deploying to a

certain hosting provider, there are all

kinds of logs that you might get from

those sessions as well. You can connect

it up to that and then you can end up

with a system where it's able to make

decisions and prioritization and then of

course do the implementation based on

not just one dimension of feedback. So

for example, if uh in my product it's

noticing that there is a particular view

that is really really slow but it now

knows that that's happening for users in

India but that I'm getting a lot of

traffic from people in India. It can