on time, right?

>> All right, we are live.

>> Great. Hello everyone. Welcome back to

day two of the Background Agent Summit.

I'm Philip and I'm here again with Will.

>> How's it going?

>> And for anyone who's new, so this is the

first-ever virtual Background Agent

Summit there is. We have in total 14

talks lined up. Yesterday was day one.

Um it was mostly about foundations and

we had seven sessions mostly focused

around infrastructure, security, context

engineering. And if you've missed it

yesterday or if you know, there were

some technical blips where the quality

of the streams wasn't as great, don't

worry, we've got it all lined up in HD.

We'll send the recordings to you via

email, so make sure to go to

backgroundagents.com/summit,

put in your email address and we'll send

all of the recordings to you. Don't

worry.

But

before we start with today's section

sessions, we wanted to do a quick

wrap-up. So a few themes kept surfacing

yesterday with that we want to pull

together. So the first one is really the

foundation matters more than the agent.

Unsurprisingly, yesterday was about

foundations, so we'll talk a bit about

that. So

um Alister from Stripe said it quite

plainly actually. He said the cloud dev

boxes they had were a strategic credit.

Um the agent worked because they had

reproducible um cloud dev environments

already. And today we have a session

from from Uber.

Um so I'll take it a bit away, but he's

also talking about DevPod, who's which

is their containerized dev environment

that they have been using for years. And

since they had these solutions in place,

it also allowed them to like start very

quickly with Background Agents, right?

Um but for most companies, the

infrastructure that they have today,

everything is running local, you have

like, you know, manual setup scripts and

all of that. And that's exactly the

reason why you um

basically land on the false summit that

we talked about in yesterday's keynote.

Um

I'm wondering um um Will, what your

thoughts were also on the foundations.

>> Well, yeah. I mean, I think that it's

really interesting to see how some

companies that already had like good

primitives there from the, you know,

sandbox, CD, dev box kind of

perspective, kind of had that leg up. Um

whereas the companies that did not had

to kind of wind up on the same approach,

uh but from the other direction, right?

So, like Cole who built OpenInspect, he

mentioned that that was the single

biggest friction point was that

environments themselves weren't actually

reproducible. And when the environment

wasn't reproducible, the agent wasn't

reproducible. So, like the the

primitives, the things that uh build up

to and wrap around the agent make more

of a difference than the tools that the

agent is calling, whether that's like

riff grep versus grep or like how it's

building out knowledge graphs and things

like that. So long as the agent can run

in an environment that's reproducible,

the agent itself is much more likely to

be reproducible. I thought that was a

really interesting take. Yeah.

And then that kind of takes us to the

next piece, right? Reproducibility is

a lot of that is coming from the world

of context. And Patrick at Tessell did a

really good talk at the end. If you

didn't have a chance to watch that, I

would suggest taking a look when we send

out the follow-up emails. Um you defined

it as like the context development life

cycle. Um and really tried to strike a

good balance of like uh generating,

testing, version controlling,

distributing, and even observing the

context aspects of the agent. What's

going into that environment, right? What

tools does it have access to? Um what is

the like most accurate, up-to-date

version of things? And then uh

from Cloudflare, Rajesh did a really

good uh talk about what they called

engineering codex, um which were

really rigid definitions of how the

agent is supposed to work and how the

applications that the agent works within

work themselves. And those codex uh

really improved the quality of the agent

themselves, right? This was all done in

backstage, which I thought was so cool.

Like there's there's this

kind of ephemeral thing that we were

doing for a really long for I guess at

this point it feels like a really long

time, but it hasn't been that long of

like IDP engineering and uh

giving developers a way to self-serve,

but all of that work has kind of

it it was nice to have as a developer,

but it's a must-have for an agent.

>> Yeah. And I think like adding to that,

so Patrick also coined the term um

context um development cycle, right? So

you really like have to be very

deliberate about how you inject context

and um and and when to refresh etc. So

it's not that simple and I think Kwak

from Genentech also made a really really

good point here that I want to um

um emphasize. So they let their agent

self-serve, right? They put it on a

nightly dream, like reflect on what it

has done and like create new skills and

like update the existing skills, and

that has generally improved the outcome.

But also to um what they observed is

that it really depends on on on what

kinds of skills, right? If it's a more

like factual context as to like always

go to these Slack channels and like

here's the ID and everything. Obviously

more context helps um because it gets

there faster, uses less tool calls. But

if it's more a diagnostic context or

like open context where you have

something around debugging, for example,

if you're a very rigid with like super

long checklist, it actually you might

end up with the same outcome quality,

but it just requires more tool calls, it

takes longer, is more expensive. So the

lesson really from him was um kind of

teach the agent what to look at but not

to what to look for. So,

I found that super super interesting as

well.

All right, let's go into the next

Sorry, do you want to add something?

>> I was saying yeah, it's it's important

not to over-engineer there. You know,

like I feel like there's still the

concept of over-engineering like an API

endpoint, like that still kind of goes

back to over-engineering the context

that you're giving an agent.

>> 100% Yeah, yeah. And it's so easy to

over-engineer with agents too cuz

producing is just so cheap.

Cool, so let's go into the next theme

that we that we

that stood out. Obviously, security was

a big one. We had two sessions around

that. So, that can't be an afterthought,

especially if you really go big on

autonomy and like don't and babysit the

agent anymore, right? So, we had Leo

Lorenzo and Chris from our own team who

talked about my prompt level guardrails,

which most of our local coding agents

rely on, keep failing. The like the

agent will just reason around them and

that's why the enforcement really has to

happen like a level below in depth. And

that's what B2 and Data Wallet doing.

They have allow and deny lists and scope

credentials and it really is like a

kernel level policy enforcement. And

we also had Stephen from Nono

from the open source side basically

sharing the same principles and that

just drove home the point that

security can't be an afterthought but if

you got it got it figured out then you

can really, you know, with good faith

let the agent run autonomously.

>> Yeah, I mean speaking of that autonomous

running,

I think Quake from from Genentech did a

really good job of talking about the

engineering that has to go into the

environment itself in order to make it

reliable enough to merge to main or to

do things where where it has a bit more

production capability. So, like

integration tests, canary deployments,

feature flags, the types of things that

you bring into

uh

production data and production systems.

If you

treat agents the same way by making the

environment the governance layer, then

you can do a lot more with your autonomy

within your you know, within your own

cloud within your background agents

ecosystem.

>> Yeah, exactly. So if if if the

environment is the governance layer, you

have the security figured out, then what

you can do is basically you can move

from software manufac- uh software

engineering to software manufacturing.

That's something that Cole Murray from

Open Inspect yesterday um kind of

coined. He was talking about, you know,

using fleets of agents like overseeing

them as an engineer. And that really

already goes into the the factory

analogy that we talked in the keynote

and we'll also hear more about later

today. And the interesting thing here is

again, if you have these two foundations

figured out, then you don't have to be a

super technical person to to work with

uh background agents because there's

like not so much that you can actually

uh do wrong. You don't even need to know

so much about security, right? So if you

have those two foundations in place,

even like PMs or designers can really um

self-serve more with agents and ship

actually code or something else. So we

see this at our at our own company where

we for example, like the slides that

you're looking at, we we did them with

Ona, right? Like and uh a PM did those.

So

um nobody that's like super technical.

So I found that super super exciting cuz

it's like very empowering if you got the

foundations figured out. Like it really

extends towards everyone else in the

company.

>> Yeah. And that that stretches so far.

Like it it's more than just even the

engineering work close to. Like the

um

with with Greg showing that genomics

could be done in a secure way accessing

like really secure classified data um

and being able to do background agent

tasks on it in a reliable secure way is

so cool. So I love seeing the potential

here, you you

Um so yesterday was primarily about the

building blocks. Um, you know, how do we

actually build these systems? How do we

design the architecture? What sorts of

trade-offs do we have to make? Um, and

what are the you know, kind of things

that we should be experimenting with and

how are those like leading to other

decisions down the road? But today is

more about how do we actually ship this

within a real organization and what

sorts of changes do this do background

agents unlock? So,

uh

Yeah, I mean the questions that we're

asking are are more along the lines of

like ROI. How do we improve the number

of uh PRs that agents generate that you

can actually like open and merge that

aren't just slop or drafts?

Um, how does that impact your review

process and things like that?

>> Yeah, and we're also going deeper into

uh what the end state could look like if

if if that's at least uh conceivable.

So, the idea of the software factory, we

talked about that multiple times. Today,

we have two talks around them. Uh and I

already saw a comment about dark

factories sounding shady and fun. So,

yeah, make sure to tune into that.

Shardul's a great guy. Um, so let's get

um quickly let's pull up the agenda and

go through so you know what to expect

and when. So, we start off with Joey

Wang from Harvey who built um who was

the engineering lead behind Specter,

their collaborative cloud agent

platform. Um, where really everything

revolves around the concept of sessions.

Then we have Lawrence at IO, Zach who

built software factory.dev, a project

where he was like running this factory

publicly for two weeks.

Um, then we have Nikhil from Uber. We

have Shardul from AWS on the dark

factories. And in the end, we have

Suhayl from um Monzo who's talking about

adoption of agents in a regulated bank

that has I think more than 3,000

microservices.

So, um really really interesting talks.

>> Well, yeah. I guess now's the time. go

ahead and kick things off for the day.

Everybody

tune in for Joey Wang and the rest of

these calls the rest of these talks.

It's going to be a really uh

star-power filled day.

>> Make sure to ask questions in the chat.

We will forward them to the to the

speakers in in case they're not live

here and we will follow up with the

answers also via email. Have fun.