Hey everyone, welcome to OpenAI Build

Hours. I'm Christine. I'm on the startup

marketing team and I'm here with Steve.

>> Hi everybody.

>> So you might might remember Steve.

Welcome back to build hours.

>> Thank you. It's great to be back

>> from our last session on responses API.

Um but today we're going to be talking

about the agents SDK.

So if this is your first build hour, uh

the purpose of build hours is to empower

you with the best practices, tools, AI

expertise to help you build with our

APIs and models. Down below on your

screen is our homepage. So you can find

all of our past sessions including

Steve's one on responses API on demand

and this will also be on demand after

our live session. So to give you a

little snapshot on what we're going to

be talking about today. So, first we'll

be going through a lot of the new

updates on agents SDK. There's now a

codec style harness um that's really

improved it and Steve will walk you

through all of these new releases. Um we

also have some new releases in the API

that we'll walk through and then we'll

move into a live demo. So, this will be

the majority of the time that we'll be

spending today. Um and we'll be building

an agentic uh task tracker and then as

always we have a lot of time set aside

for Q&A. So on the right side of your

screen, you'll see a Q&A tab. So feel

free to submit your questions. Um we

have our team who is dialing in

virtually to answer them during the

session as well as saving some to answer

live towards the end. And we'll have a

very special guest be joining us

virtually uh to help answer some of

these. So with that, I will turn it back

over to Steve to walk through some of

these slides.

>> Awesome. Thanks, Christine. Uh so great

to be back on Build Hour. My name is

Steve. I'm an engineer on the API team

here at OpenAI. First thing I want to

talk about is some of the updates we've

made to the agents SDK in the last

couple months. Um kind of uh kind of the

backstory here is that models are

getting way better at doing work over

long periods of time. Uh a lot of folks

have seen the sort of meter chart that

shows the ability of models to kind of

work uh on their own for longer and

longer periods of time that's just kind

of going continually up and to the

right. And there are a lot of great

examples of this uh even within our own

products. So many of you know and love

Codeex, our agent coding tool that you

can use to kind of build software and do

tasks from end to end. You've probably

seen it run for maybe minutes or up to

an hour. If you've used the goal

feature, maybe you've seen it run for

much longer than this internally. Folks

have gotten Codex to run for days, up to

a week on tasks. So these models are

really really able to kind of do things

over really long trajectories, which is

really cool. Um we also uh have this

sort of security agent that's built on

top of Codeex. um you know the models

are getting really really good at

finding security vulnerabilities

especially in legacy software and so we

have an agent that's powered by codecs

that actually scans repos and our our

own code and code that we depend on for

vulnerabilities that might affect the

security of our systems and then we also

have an internal data agent that's kind

of powered again by codeex that's

connected to our data links able to

answer any questions so I can say

something like how many people are using

the agents SDK how many requests were

there in the responses API two days ago

and where previously I might have spent

an hour writing that SQL query, now it's

just a simple prompt and a few minutes

later I have an answer which is which is

really awesome. Uh the reality is though

that building production grade agents in

your own systems is is still pretty

hard. Uh it's pretty hard to kind of

strike this balance between maximizing

performance and getting and having sort

of a really uh flexible maybe cross

model provider platform. um because it

more and more the models are kind of

being tailored to the individual

harnesses and so it's kind of hard to

stay in distribution while also

maximizing that flexibility and that

customization. Um there's also a lot of

challenges that come with sort of modern

computer using agents. So we think of

codeex as a coding agent uh that can do

things within a sandbox. Most most of

the time when you're running codeex that

sandbox is just your local laptop. So

those two things are together. We'll

talk about how we switched that up with

the agents SDK. But if you're deploying

agents in production, oftentimes you

want to be able to run these agents in

sort of an isolated environment and then

you have to deal with problems like the

container dies, it expires, it goes

away, what do you do? Does that thing

loadbearing? Did you have state on that

then rehydrate? Uh this stuff is is

really really hard. Uh and then kind of

the third thing is like at the end of

the day, you want to be able to have

this sort of like really customizable

framework you can use to bring your own

data in and uh maybe add your own skills

and and all of this stuff like this. And

so this combination of things is kind of

makes deploying agents in production a

little bit challenging. Uh the cool

thing about the agent SDK, we've added

we've made some updates to it, but sort

of the baseline is this open- source

customizable framework that maybe you've

used since we released the agent SDK

last March. It's been around for a

while, continues to be open source and

and super flexible. Uh we've added in

this concept of sandbox using agents and

you get to decide where these sandboxes

are run. So if you are you know maybe a

modal user or an EDB user or a

cloudflare user and you're kind of

already using those sandbox products or

maybe you're new to sandboxes and new to

sandbox using agents and you want to but

you're already using one of these

platforms we have provided first class

support for using the sandbox products

in a bunch of these uh different

platforms like ETB modal cloudflare in

addition versell uh blackel daya uh and

just things like docker and uh if you

just want to use your local laptop for

testing. Those are kind of all supported

first class. And then on top of this is

this Codex style harness. So bringing in

all of the great stuff that makes CEX

Codex, all the great tools that are in

distribution with our models, but work

great with other models as well. And

then some of the cool things like skill

use, uh, computer use, memory, and we'll

be demoing some of this stuff here in a

little bit. So uh, yeah. So basically

what the agents SDK is is it provides

this sort of model native harness that

the that the agent can use in order to

kind of do productive work over long

periods of time. So if we think about

maybe what building an agent might have

looked like you know a year a year and a

half ago where you're just kind of using

an API directly an LLM API directly you

would have done something like you know

you set up a loop you make a request to

an LLM API you get some stuff back maybe

you get a tool you execute that tool you

update the context and then you kind of

repeat until the model has no is no

longer calling any tools you might be

layering in stuff in addition to that

like web search file search MCP kind of

like building spending most of your time

building this orchestration layer around

this loop where actually you should be

spending most of the time building

product building stuff into your product

that that makes it it much better for

your end users and so agents SDK is kind

of designed to help you do all that

stuff and not really think about the

orchestration. So the agent loop is

built in again very CEX-like in its uh

design. Uh also web search, file search,

MCPS, code interpreter, skills, these

are things are all you get out of the

box or really easy to turn on. And then

we've kind of added in some of this like

sandbox using functionality that I'll

talk about uh pretty shortly. The thing

that I'm most excited about that we've

kind of done here is we've split the

harness from the compute. And uh what

does that mean basically? So if you can

kind of imagine a world where the model

is working sort of directly on a uh

computer that it itself is running on.

So if you can kind of imagine like

codecs running on your laptop is the

sort of example where the harness and

the compute are tied together. Uh

imagine putting that into production

right so you have sort of a container

where the agent is both running. So like

the the loop that is calling your LLM

API in addition to working on the same

file system those things are together.

This kind of creates a few problems when

you run this in production at scale. A

your sandboxes become all of the sudden

loadbearing and if one dies or goes away

then all of a sudden all that state is

gone and you don't have some sort of

external place where maybe you can

refresh from. In addition, you have to

sort of do interesting uh sort of

gymnastics to manage your secrets. You

ideally don't want to have any secrets

on your sandbox otherwise you might be

vulnerable to prompt injection attacks

or exfiltration. And so if you kind of

split those things up, then you can

treat the sandbox as this totally

ephemeral thing that you don't really

have to worry if it lives or dies. It

can expire, go away. And then the uh

harness, which is running maybe in a

temporal job or in AWS or you know kind

of wherever your infrastructure is

already running can sort of handle the

rehydration and snapshotting and all

this stuff. And the agents SDK is going

to make that really easy. And I'll kind

of show some examples of this. Um, so

agent SDK kind of like already fully

packed with a ton of features that you

might have already been using things

like web search, file search, uh, you

know, agent memory, text in addition to

some modalities that you've probably

been using, uh, text and voice. Um,

there are some new things that are we're

pretty excited about that we've added.

So this is kind of first class support

for skills. Uh, containers is kind of a

new thing sort of in the API, also in

the SDK. And then we've also added agent

memory so your tasks can improve over

time um and get better as they as they

go. So some of the kind of core features

here again uh code a really you know

codeex inspired harness a lot of those

same tools sandbox use so your models

can work with files in sort of a

virtualized environment you know do work

over long periods of time write bash

write python write whatever to kind of

accomplish tasks and then as always open

source super customizable

socy harness what kind of concretely

does that mean um the kind of things

we've brought from codeex into the

agency SDK are these things like uh

autoco compaction uh especially you know

this computer use through through uh the

shell. So the model is able to write

shell commands. Um we've you know the

codec style loop is this sort of async

shell interaction loop where it can

write a command maybe wait a little bit

if it doesn't finish it can go do

something else and then kind of come

back to those things and it kind of

keeps track of what commands it has

running at any given time. And so we've

kind of brought that into the agent SDK

to allow you know coding agents to work

kind of in any domain in any

infrastructure. And then sandboxes are

super crucial because you know modern

work means working over a lot of files.

If you think about kind of the work that

you and I do every day maybe you're

working in a big codebase with a bunch

of files editing files and and making

your changes. Maybe you're working with

a bunch of PDFs. Maybe you are you know

creating word documents or powerpoints.

a agents are no different and they need

access to those files and a kind of a a

a place to create those files and store

them. And so that's kind of where

sandboxes come into the picture. It's an

isolated environment where the model has

access to some files and can do stuff

and and produce uh meaningful output.

And uh kind of again agent SDK open

source and customizable. It's been open

source since day one. Super, you know,

uh model agnostic. If you want to use

other model providers, you're more than

welcome to. Anything that kind of uses

the responses API format will plug in

really well. Uh it you can make this

multi-tenant super easily. So if you

want to have a system that's running the

agents SDK where you're kind of

processing many requests from users at

once again super easy. Made session

management really simple. So agents SDK

is going to manage the snapshotting uh

rehydration of tasks. You can kind of

start from where you left off. All that

stuff is really simple. Kind of show a

demo of that. And then really

configurable. You can bring your own

tools, bring your own MCPs and and kind

of all that stuff. So I want to switch

gears a little bit and talk about some

other cool things that we've shipped in

the API in the last couple months. Uh

first one is this hosted shell tool in

the responses API. So idea here is I

want to make a just a quick API call. I

want to provide some files there that I

want the model to do something with. Um

and behind the scenes we're going to

spin up a container, load that up with

the files that you've asked us to. The

model can write code, do some stuff in

that container and then return the

results back to you. So think about this

as like a lightweight version of the

agents SDK. It's kind of this ephemeral

container. You can put a few files in

there. The model can work in that

container and then it can finish and you

can you know create these out of band.

We have a containers endpoint where you

can go and create a container, upload

files to it, spin it up, attach it to a

responses API request or you can kind of

use it in auto mode where you just add

some files and then we'll spin up the

container for you and then spin it back

down when you're done. Really easy way

to kind of get started with this, you

know, sandbox using paradigm.

We've also added the ability for you to

kind of control network access in the

containers you spin up. So if you want

to do things like domain allow lists,

lock it down entirely so you have no

egress or ingress into that container.

Uh these are kind of like all things

we've added so you have a little bit

more security control over your

containers. And we've also added the

skills API. So if anybody has used

skills in the past, there's kind of like

a pretty common problem that comes into

play which is that you kind of need a

central source of truth for these

skills. whether that's maybe a GitHub or

a bucket or now we have a skills API

where you can kind of upload your skills

too. Uh and a skill is kind of this

bundle for for those who haven't used

it. It's kind of a bundle of files that

has sort of one skill MD file that has

all the top level stuff in it, but they

can also contain scripts or other

resources that the model can use to kind

of do a pretty specific task. So, the

example I always use is like maybe you

have a tax prep skill that kind of

defines like, okay, here's all the

things that I need to know as an agent

in order to do somebody's taxes for like

tax year 2025. Has all the like IRS

rules and maybe some scripts to like

process documents and like fill in a

1040, something like that. Um, in the

with the skills API now, you could

upload that, you can iterate on it over

time and create versions. You can set up

default version and then reference those

versions with the hosted shell and all

that stuff works together super

seamlessly.

Uh yeah, so kind of a couple curl

snippets here of how you might use this

thing. Pretty easy. If you already have

zip files of your skills, you can just

kind of upload them directly to the API.

And then the last thing is we've also

shipped uh the sandbox agent stuff that

we will be talking about today in

Typescript. This is a big ask from a lot

of folks after we ship the Python

version in April. Uh sand the TypeScript

version is now available. So if you are

a TypeScript user, uh knock yourself

out. So, uh, want to switch gears a

little bit and kind of go over to the

demo. Uh, we're going to be kind of like

building a task tracker with the agents

SDK and kind of making it aic and then

offloading all the work so I don't have

to do anything anymore.

So, we're going to

Oops.

Flip over here. And uh I kind of put

this kind of like vibe coded this uh

kind of like sort of like a linear

board. Doesn't really do anything right

now, but I can kind of create tasks and

uh assign them to folks. So the sort of

idea here is that we are going to be

doing some conference planning. So you

can kind of imagine this is a tool that

I might use as a conference planner to

put together some talks. So I might have

tasks for you know creating program

assets or refining somebody's talk or

you know uploading stuff to the website

things like that. Um my goal here is not

to do any work myself. So we're going to

kind of automate a lot of this with the

agents SDK. So I'm going to create a

fake task here and I'm going to say uh

you know create program assets and we'll

say we can leave this blank and we have

a couple of things down here. where we

have we can have a assignment. Uh I need

to refresh this.

Sorry guys, I need to reset my demo.

Cool.

Okay, there we go. So I can uh I can

create a blank task here that's we can

say create program

assets

and uh I can assign this. Right now it's

Right now it's just me, but just got a

sneak peek at what might be coming in a

few minutes. Um, and I can upload files

to this. So, if I just go ahead and grab

my fake files here,

can drag these in. I can create an

issue. Uh, and in a pre-agentic world, I

would assign this to somebody. Maybe I'd

assign it to myself and I would I would

do this work, but that's not really that

much fun. So, we're going to go ahead

and create an agent to kind of help us

automate this. So, if I hop back here,

uh, we are just going to comment in this

agent definition. So, I want to take a

sec to kind of walk through what we're

looking at here. So, if you've been an

agent SDK user in the past, this will

look pretty familiar to you. Uh, the

sandbox agent is a new type of agent.

Uh, it actually is just a subclass of

the agent class that you might have been

using before. So, a lot of the same

things will still apply. A lot of the

same parameters are still going to be

there. We are having we have a couple

new things that we'll kind of talk about

in a sec, but suffice to say that we are

adding some instructions that are kind

of defined up here. Uh, and then we'll

be adding some other stuff later. So, if

I go back to my demo, refresh,

should be able to assign this to the

program editor agent. And what I'll say

is like, please take a look at these

files and uh, edit them for clarity.

I'll hit send

and then we should be able to see that

kickoff. So the first thing that's going

to happen is uh because I've configured

this first agent to use docker as its

sandbox can actually see that docker

container running and uh the agents SDK

is going to handle uploading these files

to that container. So if I pop this

might be a little hard to see but uh we

can see that a lot of these files have

now made it into the container. So the

agent is going to be sandboxed to just

this folder and we'll have access to all

of these files that it can then use to

look at. We can as we kind of look over

here, we can see that it's executing

commands. It's kind of like looking

through the files and figuring out

what's what. And then it's going to kind

of produce some output. And so it just

finished and it gave us it just kind of

gave us a little log of of what it did

and kind of told us that it's missing

some data here. That's kind of okay for

now. So,

uh, this is great, but we kind of want

to be able to customize this agent and,

uh, kind of give it some like really

specific context that is pretty

particular to what it needs to do. And a

really good way to do that is through

skills. So, prior to this demo, I kind

of put together this skill that's for a

conference, the conference program

editor. It's got some stuff in here.

Again, this is kind of entirely vibe

coded, so the actual content doesn't

matter. But what I want to point out is

that GitHub is actually a really great

place to store skills and kind of

addition to the skills API. The reason

GitHub is so great is because you can

have a lot of this a lot of these uh

kind of concerns of producing and

editing skills are kind of taken care of

for you. So you know if you want to have

version control over your skills, if you

want to have pull requests so that you

can review changes to skills, this is

all really great and a pretty common way

that people are starting to create

repositories of their skills is kind of

in Git. And we have first class support

for that in the agent SDK. So if I come

back here and I comment this stuff back

in

and hop down, we can see that we have

this uh skills capability. And I want to

pause for a sec to kind of talk about

what a capability is in the new SDK.

Capability is kind of uh this object

that is able to combine a bunch of

different concepts all into one. So if

you want to bring in uh tools, if you

want to bring in additional

instructions, maybe add some stuff to

the manifest so that a capability can

put things into the computer when it

starts up. These are kind of all things

capability can do. The sort of default

set is file system. So this is the

ability for the model to view images and

run the apply patch tool. So kind of

like edit files in line. And then we

also have the shell tool which is

actually a combination of of two tools.

This is as I mentioned earlier exact

command and write standard in. This is

this async bash command flow. And then

we have compaction which is allows the

model to compact its context and keep

going even when it exceeds the context

window. And this allows the model to

keep working for a super long period of

time, hours, weeks, technically uh

there's no limit to to how long it can

work. So these this is kind of the

default set. We're also adding the

skills capability. And then what we're

saying here is that we want to load our

skills from this git repo. And so I'm

pointing it at this git repo that I just

created. Uh we're saying we want to pull

from main. If you're not using GitHub,

you can actually change the host here,

but uh it's we're kind of like using

GitHub by default. So I'll go ahead and

save this. We'll flip back to our demo.

I'm going to ask the question, tell me

the most

interesting

thing from your skill file.

Will it send? This is going to kind of

spin back up, but I want to take a sec

now to actually talk about how we do

resume uh like pausing and resuming

behavior between sessions at the agents

SDK. When you start a sandbox agent, the

agents SDK will handle spinning up that

sandbox for you based on how you define

it. So, if I hop back to the code here,

we can go and say for by default, we're

using Docker. And so, I'm kind of just

providing these two classes here to kind

of define where I want to run the agent.

In this case, I'm saying I want to use

the Docker sandbox client. the and I can

provide some options. Different backends

might have different options. Docker, I

can provide an image. In this case, it's

just uh the Python 312 image. And uh you

can kind of easily swap this out for any

of the backends that that I mentioned

before. And we'll kind of show an

example of that in a little bit. Um but

what's cool is that when that when this

task spins down, when it stops, the

agent will automatically stop that

container and then we'll take a snapshot

of the file system and put it somewhere

you define. Right now that place is just

my laptop. So if I go over here

uh we can see some of the snapshots. So

this is kind of the a couple tasks that

I recently created. And if I untar one

of these and look, it's basically just

the exact file system that the model was

using. So when I resume that task, it is

grabbing that tarball from the place

that I defined, spinning up a new

container if the last one doesn't exist

anymore and then rehydrating that file

system onto that new container so that

it to the model looks exactly the same

as it did when it left off and the model

is actually unaware of the fact that

it's running on a new container is

totally oblivious to that. And this

makes resuming really easily easy and

seamless. And if you don't want to use

your local file system or tarballs, we

have sort of provider specific ways that

you can do this. So if you're a modal

customer and you use volumes or their

snapshot concept, this stuff plugs right

in super easily. But kind of by default,

we're using this sort of like naive

tarball approach to kind of help make

this easy across any provider. So uh we

can kind of go back and if I refresh

should be done. And it was able to kind

of like look at the skill file. If I

look at the output here, we can see that

it probably was kind of graping over

this skill file. Uh, it did quite a bit

quite a bit of stuff. So, um, but it was

able to kind of give me some output

here, kind of, uh, telling me what was

the most interesting thing here. So, now

we have an agent. It can use a skill. We

want to kind of hoist this thing up into

the cloud. We don't want to be running

this in Docker forever. Docker's great

for testing, especially on your laptop,

because just kind of works out of the

box. It uh is a really great place to

kind of do local development, but it's a

bit hard to deploy in production.

There's a bunch of companies that have

built really great first class agent

sandbox products that you can use that

we've added first class support for. In

this demo, I'm going to be using modal,

but you could also be using Cloudflare,

ETB, Verscell, Daytona, Blackel, uh or

you know any of the other you can kind

of bring your own implementation if if

you choose to. So I've already set up a

uh modal kind of app here for us. So I'm

going to go back to my

uh code here and we'll go to step three

and we will enable modal and here we're

kind of returning this sandbox provider.

Um but the kind of the key thing here is

just this client and options which are

the things that the the agents SDK

expects and here we're just kind of

providing this modal sandbox client. If

I go look at the definition of this

thing, uh this just inherits from base

sandbox client just like the docker

version and we are saying here that uh

we are going to use the same image. So

that same Python 312 image and then uh

in addition we're going to be using R2

for snapshots. So that both our

snapshots are now off my laptop and into

the cloud. Additionally, the modal

sandbox is going to be moving into the

cloud as well. So we'll kind of show an

example of what this looks like. So save

all this. Go back to my

here and we'll say uh edit these files

for clarity.

Go back,

load this,

and we'll assign it to our program

editor and we'll say get started,

please.

And so now we're starting it. We can see

that we're starting a modal sandbox

instead. And if I hop over to modal, I

can see that now we have a sandbox uh

that's starting. Zoom in a little bit if

that's hard to see. And this will take

uh usually they start up in just a few

seconds or around a second. So now it's

running. What's happening behind the

scenes is the agent SDK is handling sort

of the applying all of the files that we

have defined onto that sandbox. And so

want to take a sec now to actually go

and look at this manifest object that we

talked about. And a manifest is is

pretty simple in concept. You're

basically saying this is what I want the

file system to look like when the agent

spins up. Here's how I kind of attach

files to this task in a way that I can

share across tasks or share across

agents. So manifest is just kind of a

simple class. Um we're saying the

entries are this tree here basically

kind of defining a directory structure

and this this is a pretty simple

example. So we're just saying we want to

kind of establish this base directory

structure and then we'll upload files to

the sandbox directly. But you could also

here plug in things like files if you

want to copy things from, you know,

wherever the harness is running. Or if

you want to plug in an R2 bucket, S3

bucket, an Azure blob store account, all

these things are fair game, a GitHub

repo, kind of a whole flexible set of

primitives here that allow you to put

the files on the container that the

model needs in order to kind of get

started. And what happens is that we

render a version of the file tree to the

model when it starts up so that it can

kind of see an example of its workspace.

Uh so that it doesn't really have to do

too much grepping or lsing to understand

what's on the file system. And so you

can kind of add descriptions to these

things too, but we won't get too far

into that. Um so now if I go back to the

demo, we can see that this finished. Uh

and what I want to show here is that now

if I go to my Cloudflare account and

refresh, we can see that we now have

snapshots that are stored in the cloud.

So if I want to resume this task and I

dep have this app deployed in

production, it's really easy for the the

agent to just kind of like pull its

context for the agent to pull its

context and the snapshot from R2 instead

of you know anywhere else. Uh it just

provides like a fast pretty clear

interface for this and so just want to

give a quick demo of that. So now uh we

kind of want to we have this agent it's

deployed in the cloud. It's pulling its

snapshots from another cloud provider

and we want to go ahead and add our own

tools into it because you know bringing

our own context and our own

functionality into agents is is pretty

critical. So if I go back to my program

editor agent, I'm going to comment this

back in

and we have some predefined tools that

we built sort of before this demo.

There's there's three of them. We have

one for updating the status of a task.

We have one for updating the assenee of

a task. And then we have one for

searching all of the assignees. So, uh,

if you've used the agents SDK before,

function tools work exactly the same way

as they always have. They're essentially

just Python functions or in TypeScript,

TypeScript functions, and you decorate

them using this function tool decorator.

And what this does is it allows us to

translate this function call into sort

of the API representation of the

function tool with all the parameters

uh, clearly defined so the model knows

what to call. And then when the agents

SDK receives from the API an instruction

to call a specific tool, it routes it

automatically to this function, calls

this function and then sends the

response right back. So you don't have

to sort of like be implementing this

function calling loop yourself. Uh and

so we will uh just show a quick demo of

this. I'll go back here and I'll say um

please assign this

task to Steve and mark it as ready for

review.

Let's send. While this is spinning up,

I'll kind of cover some of the other

cool things you can do with the function

tool. Um kind of a bunch of things you

can do here. So if you want to kind of

override the name, like what the model

sees is different from what the name of

the function is, you can do that. You

can provide a description to kind of

help steer the model to how to use this

tool. You can provide guard rails around

uh whether you know if the model's call

trying to call the tool with specific

arguments. You can add hooks around that

to kind of you know control the

execution flow. You can add timeouts and

things like this which is which is

pretty cool. Um and you can kind of like

set whether it's enabled kind of

dynamically maybe based on a feature

flag or something else. Uh so you don't

have to like comment and comment out

these these lists of tools which is

which is pretty cool. So go back to our

demo here

and we should see it uh assign the task

to me which is cool. So we can kind of

see like using these tools and this is a

great way to sort of bring your own

context or your own functionality from

your app into your agent whether maybe

you have want to give it access to your

database or want to have it be able to

you know uh use Slack through an MCP or

something like this. These are kind of

like great ways to bring in your own

functionality into the SDK and into your

agent. So, uh, now we are going to talk

a little bit about like tool call

approvals. So, ideally, um, there are

certain tools that maybe kind of require

a human of the loop step in order for

you to have confidence that it's kind of

doing what you ask and not going off the

rails. If you have been a Codex user in

the past, you're pretty familiar with

this. It tends to ask you a lot, do you

want me to run this bash command or am I

allowed to access the internet? This

sort of thing. You can do this in the

agent SDK pretty easily by just

providing a function. Uh you can I can

either say you know true directly or I

can provide a function that returns true

or false dynamically based on the

parameters. And so in this example we've

said if you're trying to set the status

to done then it needs my approval first.

I'm not going to let you mark something

as done without taking a quick look at

it first. And so if I refresh this and

go back and we'll go back to this other

task we have and I'll say please mark

this task as done.

We'll send that and this is going to

spin up a sandbox rehydrate the state.

Uh it will kind of like examine its

current state and then it should call

the tool and then we should kind of see

a tool call approval um pop over and

over here. Uh while this is working

another thing we can kind of show is the

ability for the agents to kind of hand

off to each other. So let's say I want

to, you know, I kind of have this

built-in mechanism here where we have

the ability to search and update the

assigne. And this is great for agents to

kind of work together. So I have two

agents now in my system. I have one for

producing assets. Um, and then I have

one for just like the program editor

agent that we looked at earlier. And so

I can go ahead and create an issue new

issue here and say like uh please

um

refine and uh

this content

and produce some assets. We'll hit uh

drop some files in our same set and then

we'll assign this to the program editor

agent and we'll say like please uh

refine this content

and then assign to the what's the other

one called asset producer agent

and we'll create that. Then we'll kick

this task off. the program editor agent

will be the first one to pick it up. It

will kind of do its thing and then it

will uh reassign this to the asset

producer agent and then that agent will

kind of kick off and then you know be

able to create some assets and then put

them back in the sandbox. And so kind of

coming back to my uh previous task here

it looks like this marked as done. If I

go to activity here I can see that we

now have this request uh for approval.

And if I go ahead and approve this, then

this will kind of kick the flow back off

and then it will eventually kind of get

marked as as done here.

So, uh let's see. I think the last thing

I want to show here is sort of the

ability to

to mount uh external buckets to your

manifests in a way that uh allows you to

kind of bring in external data. So, what

are the reasons you might want to do

this? Let's say you have either a ton of

files, let's say you have like hundreds

of PDFs that you need the model to be

able to work through and it's kind of

annoying to just, you know, have to copy

all of those to a sandbox every time you

want to spin it up and have it do some

work. Or let's say you have data with a

a strong freshness constraint, so the

data is changing all the time and you

want the agents to be able to work over

data that is like super live. In either

one of these cases, mounting data from a

centralized source of truth instead of

copying it in where it might already be

out of date by the time the agent starts

working is pretty critical. And we've

built that natively into the agents SDK.

So if I go to

uh

if I go here, what I'm going to do is

kind of comment this in. And we've built

this uh R2 attachment store. And so this

is kind of doing a couple things. This

is a taking our uploaded files and then

instead of just putting them on my

laptop and then copying them to the

container, we're actually now uploading

them straight to R2. So that's going to

be our source of truth just for our

application layer. And then back in the

manifest, oops,

uh we're saying if we are using this R2

attachment store, then we're actually

going to use a different manifest here.

And we have a similar layout. So we have

task and then these four folders. But

instead of the input being an empty

directory that we then copy files into,

we're instead saying I want to use this

S3 mount type. And we're technically

using an R2 bucket, but we're using the

S3 mount. The reason is is because R2

and S3 are API compatible. So we have

one object for them both. But as we look

through the parameters here, we can see

we're using this R2 bucket name passing

passing we're passing the access key and

secret key. And then we're using this uh

modal cloud bucket strategy. A lot of

these sandbox providers, modal included,

have native ways to mount external

volumes to their sandboxes. And so we're

using the sort of built-in one for

modal. But if you're using a Docker

container or sort of any generic

container, we provide a couple out of

the box. You can either use our clone or

fuse. And these both work equally well.

And it will depend a little bit on your

systems which one will work better for

you, but they're both provided out of

the box. So pretty easy to just plug

this stuff in. And so now uh I will go

and create another issue here. We'll say

edit these assets again please. And

we'll grab our files.

Drop these in.

Hit create.

And

we should see so this has been I didn't

assign this but we'll sign in a sec. Uh

what I want to show here though is now

we have if we go back to our bucket we

now have tasks uh and we have a a

reference to our task here that has all

the files in it. So now if I go back and

then assign it to our program editor,

program editor and say get started,

hit send, it's going to be able to we'll

watch this thing spin up and then it

will be able to read and write to this

volume just like it would if it were on

the file system local to the container.

So you can kind of bring in these places

of context of you know file storage from

kind of anywhere across the internet and

the model will be able to access it as

if it were on the local file system.

It's completely opaque to the model that

it's actually over operating over the

network. So there are some trade-offs

here. You might incur some additional

latency as the model is kind of like

lsing maybe a large bucket that takes

you know a long time to list all the

files through. Uh, but there's sort of

this, you know, really nice trade-off

that you get where you get to, you know,

maintain your freshness constraint or

have it work over a ton of files without

having to sort of like copy all them all

onto the container where you might be

sort of like limited in space. Um, so

cool. We can wait a sec for it to see

the spin up, but that kind of brings us

to the end of the demo here.

Uh, you know, we kind of covered a bunch

of cool things. Uh we showed how we can

build agents that both inspect files,

spin up sandboxes, and kind of any

environment that you choose, maybe one

that you're already using. Uh agents

that can edit code and kind of like work

on these long horizon tasks for maybe

minutes or hours or longer uh over, you

know, whatever period of time that you

want and whatever context you want. So

yeah, with that, I think we'll do some

questions.

>> Yeah.

>> Cool.

>> Okay. Um next slide. I'm really excited

to welcome Nish on stage with us. So

Nish is product for agents. Nish, do you

want to give a quick intro?

>> Hey guys. Uh yes. So product manager on

agents. Uh really excited about the

stuff we're doing with agents SDK. I've

been answering some of the questions in

the channel. Really great questions and

uh yeah, excited to get into it.

>> Awesome. Okay. Um next slide here.

Perfect. Um, so the first question, when

are harnessed based agents best versus

building with just responses API?

>> Cool. Yeah. So I think that if you the

responses API is great for a huge

variety of tasks, you know, I think the

places where it really shines is and

it's an agentic model by default and or

agentic API by default and we kind of

talked about that in the responses API

build hour from October. um it really

shines I think if you are using if maybe

you're using a language where you can't

find a really good uh sort of agentic

framework that you like. So maybe you're

building an elixir, maybe you're

building in HASLL or something a little

bit more say esoteric, you can always

just build on the API. You can kind of

recreate a lot of these concepts

yourself. It might be a little bit more

work. Um the you know sort of like

harnesses are becoming uh sort of really

key to how the models operate. So in

particular, the codeex harness with

openAI models is sort of like becoming

more and more coupled as time moves on.

And so you know in the future, it's very

very possible that you will be able to

get the best performance from the

harnesses that the models are trained

with. And uh so that's where kind of the

harnesses are are really going to shine.

Um and also if you you are using uh an

environment where you can kind of drop

these things in, it just is making if

you're building agents, it's going to

make a lot of this stuff super easy. Um,

however, if you're doing, you know, kind

of oneshot tasks like maybe a

translation use case where you just need

to like just oneshot translate a

document or, you know, taking

unstructured data and structuring it

into JSON or some other format,

sponsor's API just out of the box is

going to be really good at that stuff.

And you can use all the hosted tools to

kind of have these like really light

agentic loops that are uh pretty easy to

to orchestrate in just one API call.

>> Cool.

>> Okay, next question. Does the agents SDK

feature any built-in out ofthe-box

persistent state management to handle

this pause and resume behavior natively?

>> Yeah, totally. So, I showcase a little

bit that snapshot mechanism. That's

totally out of the box. That works if

you just start using it. Um, the only

thing you have to define is kind of

where you want to store the snapshots,

whether it's in R2 as I mentioned or

just kind of local to the to disk which

is the default. Um, and so this is kind

of that, you know, really out of the box

thing where it will take one sandbox's

file system and then zip it up and then

put it somewhere else. Uh, so that

another sandbox can resume from the

point where it left off. Um, but also

the run state is sort of managed. So you

can think about two pieces of state

here. It's kind of like the file system

and then all of the messages in that

conversation so far that the agent will

need to continue from agents SDK manages

handling kind of like both of those

pieces in in one shot. And so you can

take that thing that has both a pointer

to where the snapshots stored and then

also the full roll out and then you can

store in your database. It's just a JSON

object and then load it back up later

and then you can kind of resume

losslessly maybe if you have sort of a

multi-node system or something like this

that just has the database to share

state.

>> Nish anything to add there or we can

move on to next question. Yeah. No, I

think that covers it. Like essentially

we store the roll out and we also

snapshot the file system which is uh a

little bit unique compared to other

SDKs. Um so you should be able to uh

resume very like long running uh dayong

agents pretty easily.

>> Cool. Okay. Next question. How would you

compare the code executing harness in

the agents SDK via the sandbox agent

versus the actual codeex harness? Any

capability? Uh Nish, you want to take

this one or?

>> Yeah. Yeah. Um so there are some sort of

small uh differences between the sort of

like actual codeex binary and the agents

SDK. Agents SDK does come with a lot of

those core capabilities. Uh some of

those are like the bash tool. Uh the way

codeex edits files, skills, memories. Um

there are some things in the codeex

binary that uh we are still going to

like build into agents SDK eventually.

uh the codex binary is like this sort of

like multi-threaded thing that you can

run uh multiple different sub aents in

agents SDK we have this concept of

handoffs so you can do something like

very similar in agents SDK you should be

able to do like a lot of the things that

you can do in the codex binary and

agents SDK um but yeah ultimately like

agents SDK is like very much uh in

distribution uh through the Codex

harness today.

>> Awesome. Thanks Nish. Um, next question.

Can agents persist and restore

longrunning workflows automatically?

>> Yeah, definitely. This is kind of what I

showed in in the demo. It's kind of like

very easy with very little. We didn't

look at a lot of my orchestration code,

but there's not really that much to it.

Just kind of kicks off the SDK, stores

the the roll out, that JSON file in a

file just on the laptop, and then is

able to kind of resume uh pretty easily.

So yeah, I think mostly most of the sort

of engineering that you'll do is

figuring out what tools to bring in,

what what skills should I add to the

agent and uh the sort of orchestration

of starting and stopping and resuming

tasks is is sort of uh just out of the

box.

>> Okay. Can the sandbox get auto destroyed

after the task is completed or is it a

better pattern to keep the sandbox

running?

>> Yeah, this is a good this is a great

question. Uh so I I actually only demoed

one mode of the ages SDK. But there's

two ways that you can use sandboxes. You

can do the sort of what we call um you

know SDKowned sandbox where when you

spin the task up if you just provide it

the recipe for creating the sandbox. So

you provide it one of the clients like a

docker client or an ETB client it will

handle spinning up that sandbox for you

and then when the when the turn is done

it will spin that sandbox down. But you

can also eagerly load a sandbox. So you

can kind of create a sandbox out of band

and then pass the reference of the

sandbox to the agent and then that's

what we call like user own sandbox and

so you would handle managing the

lifetime of that thing. It the SDK won't

automatically spin it down at the end of

the turn. So you can kind of keep it

warm and then use it across like

multiple turns instead of spinning it up

and spinning it down. Um, but the out

of- thebox behavior is that we'll handle

spinning it up and and and down so that

you don't end up in a situation where

you've built your demo and then all of a

sudden you have 100 sandboxes running in

the cloud that you're paying for and

it's kind of providing a bad outcome. So

there's a bunch of different ways that

you can use this. Um, but yeah, by

default we the HSDK will will handle the

lifetime.

>> Hey, how should I populate and persist

my file system? Yeah.

>> Yeah. Happy to take this one. Um, so

essentially we have this concept called

the manifest uh in agents SDK. Uh, the

thing that I really like about the

manifest concept is you can kind of

choose how you want to populate and

persist uh the file system into the

sandbox. Uh, so you can essentially copy

over all of your files from cloud

storage into the sandbox or you can

mount uh, you know, all of those files

uh, onto the sandbox. And there's sort

of like different benefits of of either

if you copy over everything like during

run the runtime the the model will be

able to read the files a lot more

quickly but you know startup will be

slower because you are copying like sort

of this big blob uh into the sandbox and

there's sort of the second world where

you can optionally mount something on

the sandbox and uh startup will be like

uh very very fast but like over the

runtime it has to like read these files

over the network. Um, but the nice thing

about agents SDK is we kind of like let

you do multiple different ways to kind

kind of like mount or copy over things

in the sandbox and then persist that uh

after the sandbox spins down.

>> Awesome. Thanks, Nish.

>> Okay, last question here. Can a

supervisor agent monitor and coordinate

hundreds of specialized agents

simultaneously?

>> I think the answer is yes. It would

definitely it would take a little bit of

work to kind of spin this up. And we

have some uh cool stuff around sort of

like multi- aent frameworks that's kind

of coming in the near like weeks and

months. Um so it will get a lot easier

to do this in the future, but uh there's

nothing kind of stopping you from doing

this now. You can kind of, you know, go

crazy with agents and spin up a ton and

then have one that can kind of check on

the status between them. I think like

ways we've seen good coordination

patterns is either by through messages.

You can give the parent agent a tool

that allows it to understand what sub

aents are running and then check in on

them. So kind of like check in on their

progress or you can sort of like have

them all communicate via the same file

system or like a database or something

like this with kind of a variety of

communication mechanisms you can use to

coordinate them. But yeah, nothing

stopping you from from doing this. And

um you know, I think that this sort of

massively parallel work will become sort

of the default in the near future.

>> Love that. Okay, so let's wrap up with

some resources. So on the screen here

are everything you need to know to kind

of get started with agents SDK. Um we

had a ton of questions also in the chat

about your demo and where they can where

they can find this. So on the very

bottom is our build hour repo. Um we

have all the code from the last build

hours and we'll be we'll be posting that

um very shortly right after right after

we're off camera. Um then for any

upcoming build hours, feel free to check

back on our homepage. Um so if you hit

the next slide, we show you just the

link again. Um and after this session,

we'll email out a survey. Let us know

what topics you want to hear about next.

And we'll also include the link for this

recording so you can catch up if you

missed the beginning. Um you can catch

get up to speed um there. And that's all

for now. Thanks so much for joining us

and we'll see you next time.