I don't think it's crazy to say that the

people who know how to run teams of AI

agents are going to be able to

outperform everyone else in this AI age

we're in. So, how do you actually run

teams of AI agents? What does this mean?

What's the stepbystep way to actually do

this? And what are the tools you need to

know like Devon AI to actually go and do

this? Well, on today's episode, I

brought on Ryan Carson, and he clearly

explains all of this for you. even gives

you a tour of his insane setup for how

he manages uh his AI agent. So, by the

end of this episode, you're going to

understand what cloud agents are, how

you can set up automations, how you can

create a software factory that creates

software 24/7, and you're going to be

able to understand it just really

clearly because he does such a good job

at explaining all this. So, enjoy the

episode. I'll see you at the end, and

have a creative [music] day.

Ryan Carson back on the pod by the end

of this episode. Ryan, what are people

going to get out of this?

>> So, you're going to do be able to do

three things really well. You're going

to be able to run cloud agents. You're

going to be able to do automations. Uh,

and you're going to be able to ship

faster. So whether you're a founder, a

soloreneur, a stay a stay-at-home

parent, you're just going to become an

agent pro by the time we're done.

>> Okay? Cuz I think a lot of people

listening to this might have they've

played with agents, cloud agents. Um,

but you're going to help them just get

the confidence basically to be world

class. You know, I think that's what I'm

trying to that's what I've been thinking

a lot about. It's like okay there's you

know a bunch of people who have played

with agents but how do we become

worldclass agent operators?

>> Yeah essentially you are a manager of

agents now. So what no matter what you

used to do, whether it was a people

manager, an IC, you are going to become

a manager of agents now and you need to

be the best in the world. And it doesn't

matter whether you're a VC, a founder,

uh an IC, a mom, a dad, like a college

student, you should manage agents and be

the best at it. And we're going to show

you some tips for that.

>> Okay. So where do we start?

>> Okay, so let me give some context,

right? So people understand where is

this coming from. So, I've been a

founder CEO for like 25 years. Um, uh,

I'm working on my fourth company now.

Um, at my, uh, last company, Treehouse,

which was acquired, we had, you know, I

think up to like 110 full-time

employees, taught a million people how

to code, you know, so I've done the

whole build organization thing and

manage organization thing. Um, and you

learn a lot about managing people on

that. Um, and now this startup which I'm

running it, I am one employee right now.

Um, so we've raised a seed round. We're

scaling really fast. Our revenue

probably is going to 4x this month. Um,

and I'm probably going to be hiring my

first employees. Um, we are essentially

an AI divorce agent for divorce firms.

So, uh, we we we do the the dirty topic

that no one seems to want to want to

tackle, which is family law. So, I've

learned a lot about how to scale me,

right? Um, and so I'm going to help

folks understand how do you manage a lot

of agents well? Um, how do you check the

work? How do you automate things? How do

you work from your phone? Um, you know,

I do almost 50% probably more of my work

from my phone. Um, and uh, it's pretty

wild. So, that's where we're going to

take you. And and and I will say as

well, like this is going to change every

3 to 6 months. Um, so, uh, we're going

to share a lot with you. Get your hands

dirty. Try it. Um, and, uh, the best way

by to learn is to do. Cool. Yeah, I'm

actually I'm particularly interested in

the phone stuff because I actually do

zero% of my stuff on the phone.

>> What? Yeah. Okay, let's [laughter] talk.

>> So, uh let me start by just showing you

well when I'm at my desk, what is the

world like? Right. So, right now I'm in

my home office. Um I I I I work from

Connecticut. Um and I do all my desk

work here and I've got this big 52-in

Dell monitor. Um, I used to have like a

couple monitors and I was like, screw

it. I just want a huge monitor. So, uh,

let me share my screen and, uh, all

right. So, what I do is I set up eight

screens on my monitor and, uh, I also,

uh, enjoy food, uh, and snacks as as I

code. Um, but the key is to have eight

things up at once because I have to

multitask a lot of agents at once. Um, I

also use a paper system to keep track of

some of my to-dos, which is a little bit

weird. I use the UG Monk system, which

is kind of fun. And I've got a killer

Razor mouse. Um, I was getting a lot of

wrist pain, actually. Um, so I decided

to go uh vertical mouse on that. Um, I

also uh have a button uh which actually

is connected to Whisper Flow. So, I

almost uh use Whisper Flow for all of uh

all of my stuff. So, I've got Slack in

the top left. That's where I get

notifications and things about what my

agents are doing or what is happening.

Um that that's the primary thing I use

Slack for. Uh and then what I do is I

lay out uh various other screens. So

I've got a one password always set up.

It's very important to keep your keys uh

safe, secure, and separated from the

agent. Right? So I've got all my prod

write keys in one password. I do not

give my agents uh prod keys. Um, so

production writing is dangerous, right?

And your agents will do something bad.

Um, and so you need to set up a system

where they uh ask you for the key when

they need to write to prod and then you

go into one password, you copy and paste

it into the session and you're both very

clear about what you're doing. So uh

that's one tip. Um, keep your keys

secure. And then I use Devon. So uh

Devon is is created by Cognition. I

think it's one of the best software

factories in the industry. It's not

cheap um but it's good. Um the things

I'm going to talk about today are are

not Devon specific. Uh but they do work

very well in Devon. And one of the keys

is cloud agents. Um then I usually have

two screens set up where I'm testing um

my app. It's just a lot of testing. And

then of course I've got X in the top

left and top right. And then on the

right I've got codeex um which we'll

talk a little bit more how I do most my

work in in cloud in Devon. Uh, and a lot

of that work, like we talked about, 50%

is probably on my phone. Um, and then

whenever I do desktop or whenever I'm

doing, uh, things locally, I'll do it in

codecs. Uh, usually that's in the bottom

right. So, that's kind of my when I'm at

my desk, uh, my setup, um, and what I do

when I'm here. But the truth is I do a

lot of work, you know, from this my

iPhone. This is my beautiful wife. Um,

uh, it's just I get a lot of stuff done.

You know, I I'll be in the shower and

I'm thinking, okay, I need to check on

that PR or maybe I want to land that PR,

right? You know, grab it, talk to Devon,

which is just done in a browser on my on

my mobile and I get a lot of stuff done

that way. So, that's my setup.

So I I keep hearing this that Devon is

is almost you know it's the most

powerful coding agent and you know can

you just for folks who haven't heard of

Devon or or who haven't played with it

like why is that what is going on? So,

um, so about 2 years ago, maybe two and

a half years ago, Devon came out and

they pitched it as this idea of like

it's a software engineer, right? And

honestly, it didn't work very well,

right? The the models were just not good

enough. And no matter how good the

harness was, the models just weren't

there. Um, and then fast forward to, you

know, 2026, um, the models are

definitely good enough. And when you

have a good harness that is cloud-based,

um, it's really good. So, think of it as

you've got your choices, right? So, if

you're using agents to build things,

you're probably using codecs, you know,

from OpenAI or using Cloud Code from

Anthropic or you're using one of the

indie, I call them the the indie uh,

agent harnesses. You've got AMP, you've

got Cursor, which is not independent

anymore, but sort of own owned by Elon.

Um, and then you've got cognition which

makes Devon. You've got factory uh as

another one as well. So you've got these

choices, right? And so I thought, okay,

I want to scale myself horizontally.

Like I want to be able to run as many

agents as I possibly can and I don't

want to be working on my laptop and

trying to figure out

which code is colliding, you know, which

work tree am I on? all a lot of

technical stuff that really

[clears throat] it's it keeps you from

shipping. Um, and you know, I know

there's a lot of different folks that

listen to the show. You might have

soloreneurs who are are are newer to

coding. They don't have computer science

degrees. They're not super technical,

but they're building things. Then you

may have, you know, software engineers

who are multiplying themselves 10 to

100x. They they they have deep

understanding of software and

architecture. And then you have founders

who are coding but are abstracting

themselves. and you have VCs who are

coding, you know. Um, but, uh,

everybody's going to learn a couple

skills. Um, and what's interesting is

I'll sort of, uh, zoom out for a second.

I think we all thought that engineering

and being technical was going away, but

actually what's happening is the more

you become a better agent manager, um,

actually the more technical you become.

uh and I I think it a good analogy is

that you are basically an engineering

manager right and so to be a good

manager of many agents you do need to

become technical right there's this

reality of you have to understand what

Postgres is you have to understand a

production environment versus a dev

environment you have to understand

migrations like but the truth is you'll

learn all that by using agents right and

so I am way more technical than I have

ever been in my whole life. Um, and so I

think the idea is become more and more

technical. Therefore, okay, so let's

back up. Why Devon? What is it? It's a

really good agent. They have uh their

cloud environment really nailed down. So

that's the first takeaway I want to talk

about today is cloud. Okay. Um, how can

you run more agents, get more done? Uh,

and the answer is you work in the cloud,

not on your local machine. Now,

for the folks that aren't super

technical, I'll sort of explain what

does that mean. So, when you're writing

code or you're building things, you are

editing code, right? That code has to be

edited and then hosted somewhere, right?

Uh and and usually what we used to do is

we used to all work on our Macs and

that's called local development. And you

have to set up your whole local

development environment. You have to

have your database, you have to have

your authentication, you have to have

everything local, you have to have your

dev server set up. Um, and then what

happens if you want to do two things at

once? You have to either use get work

trees, which is pretty technical and

hard to understand, or you have to

literally make another copy of your code

in another directory. And then, so then

to work on two things at once, you have

to have two copies of your code, which

is hard. And then what about three or

four? What if you want to do five or 10

things? It just doesn't work. So what

Devon does and and I think a lot of good

uh independent agent harnesses like

cursor is doing this now um I think

codeex is starting to do this okay is

they create what's called a VM or a

virtual machine in the cloud and it's

it's basically a little server and it is

your development environment and you

click a button and it spins up and all

of a sudden you can code in there and

the beautiful thing is if you can you

can create infinite numbers of these

things. So I often have, you know, at

least five uh uh cloud agents working at

once, often 10. And there's no risk that

the code is going to collide. So what it

does is it takes away all of the mental

overhead of trying to coordinate these

things. Like I want to work in this part

of the code, but what if it overlaps

this part of the code or what if, you

know, I want to do that? Have I

synchronized and pulled the latest? like

I don't know. And when I work in the

cloud, I literally just open up my

browser and I click new session and a

new and a new VM spins up and I never

ever have to think about it. So I think

if you are working locally, I honestly

think you are a caveman. Like I I think

you're you are holding yourself back and

you are shipping 10x less than you could

be. And it is not smart. No matter if

you think the cool people work locally

and that all the smart ends, it's not

true anymore. You work in the cloud.

>> Yeah. I mean, there are a lot of people

like on X who are sort of like local

maxis who look at people

who spin up virtual machines and cloud

agents as sort of like amateur-ish. What

do you say to people like that?

>> They they are not they are not doing

real work.

>> [laughter]

>> Um, here's the deal, right? So, as soon

as we hit PMF with Untangle and we and

we started acquiring real customers and

having to ship real fixes, real

features, like I had to

probably 50x my output, right? Um, and

you just can't do it if you're working

locally. So, I think it's it's nice to

work locally and sometimes you have to

if you're doing heavy front-end stuff.

So, say say you're shipping a brand new

feature with a lot of UI, like a lot of

new UI, right? Then yes, of course,

you're probably going to spin up a local

agent and you're going to do some light

front end, you know, probably

wireframing. Um, but as quickly as you

can, you want to move that into the

cloud and then let a cloud agent take

over. Um, so I think this is happening

to a lot of people. I think a lot of the

traditional

knowledge of how to work is is is out of

date. Um that's where we're at.

>> Okay. So if you do start using cloud

agents and you do 50x output, you know,

in some ways that is a little stressful

because you have so much stuff going on,

right? And now all of a sudden you have

to manage more. So how do you how do you

structure a system so you don't go

absolutely crazy?

>> So it's interesting I think first of all

you have to shift your expectation of

what your work is and what your work is

now is making high stakes decisions

almost all day. So in the past like we

all had the luxury of making probably

you know at most maybe two to three

highstakes decisions per day right you'd

have an important meeting with your key

team or with a customer and you'd make a

couple important decisions and you would

communicate those decisions and then

they would get done over you know two

weeks. Um, you know, I think you have to

sort of mentally pace yourself to say

probably by lunch you're going to have

made, you know, 10 to 20 high stakes

decisions.

Um, and so you have to learn how to do

that. So, which is your question, how do

you do that? Um, and I think there's a

couple simple things, right? Um, so when

I'm working with my agents, I actually

have to organize my threads by and I pin

the ones that are the most important

work we're getting done today, right?

Cuz there's a lot of quick buck bug

fixes or, you know, small things you're

shipping and you kind of learn to

separate two into two buckets buckets.

What are the big big important things

I'm getting done today?

Pin those. And and then the other things

just let them rip and get back to them

when you can. Um, and then set yourself

this kind of uh almost timer, right,

where you're like, "Okay, I'm going to

go crazy if I just click through my

threads like this all day." You just get

completely wiped out, right? And so,

it's almost this discipline of saying,

"I'm actually, you know, that the

phrasing smooth is uh slow is smooth and

and smooth is fast."

>> Yeah. I I think there's a little bit of

of that where you realize I can only

check on my agents and make highstake

decisions probably every 25 minutes.

Um and so what you might do is settle

into this cadence where you check on

your high stakes threads like every 25

minutes just so you can like literally

mentally rest. Um uh and then the other

side is that is just understand that

this is the world we live in and it is

kind of exhausting. Um uh and and that

is kind of where we're at. And in in

order to survive in this new world,

you're you're going to work a lot more,

not a lot less. And on that note, I'll

share my screen because there is

essentially

uh a world we live in now where you're

going to ship probably,

you know, so this is

my PRs, right? And we're looking, these

are days, right? And you can see, you

know, the average here is is sort of

this 22 to 25 PRs a day, right? And and

sometimes, you know, 40 a day. Um most

of these are being, you know, merged,

right? A couple are being closed. And

what's funny is what happened this day.

Um I went uh hiking with my son.

>> Which day is that? I can't see your

cursor.

>> Oh, it's uh the was the 18th.

>> 18th. Um, and so we went to climb Mount

Washington and I literally couldn't

access my my phone. Um, but notice

immediately the morning of, right? You

know, while he was sleeping, I I shipped

what is that, you know, eight PRs.

[laughter]

Um, so this is kind of, you know, the

world we live in now. Um,

and we could talk about that. Um, but

let's zoom out to the the lessons here.

You've got to go cloud, right? You have

to understand in order to compete in

this new world, you're going to be

working half the time from your phone.

And it is tiring. Um but but but it's

like going to the gym. Okay, I have this

new muscle like I never had to exercise

before and it turns out this weird back

muscle. It turns out like that's the

most important muscle in the world.

[laughter]

And if you don't do it, you're not going

to win. So the way to do that is then

set some sort of cadence for yourself.

Pin your most important threads. Write

it down. So, this, you know, what I'm

going to sort of flash up on the screen

here is my written to-do list. Um, every

day, you know, I use this hilarious

analog system to like to remind myself

what are the most important things we're

going to ship today because there's

going to be 10 probably PRs that we ship

that are fire, you know, putting out

fires. Um, and and you can get

distracted and exhausted. So, so cloud

and then exercise this new weird back

muscle which is making high stakes

decisions all day. Um,

>> including on your phone, right? What

you're basically saying is like

>> you're going to because of the velocity

of

uh

of you know do knowledge work and you're

you're just you'll have [sighs]

>> stuff going on and you will need to be

there to say yes. know actually this,

you know, giving feedback in real time

and

>> you know why your point is why wait

until you're in front of your Mac to

give feedback.

um

>> it makes no sense

>> if you're trying to hit product market

fit and if you're trying to 4x revenue

in you know in a month or whatever it is

um you know why

>> got to be available

>> got to be available

>> it and you know I think it just it's

easy to think of these things as humans

right like you have a team now that is

infinitely scalable they're going to do

work very fast and they're going to be

blocked by you making important

decisions right um and you can't

delegate the decisions Yet I think we're

still very much even if you use Fable

like this is a trick is is use a Fable

thread as kind of the manager and then

ask it to spin up children and I do this

a lot. So, you know, I'll spin up a

Fable thread and I'll say, "Okay, um,

you know, we want to accomplish these

five things. You know, spin up five, you

know, child Devon sessions. Don't use

Fable. I use Fusion for that, which is a

cheaper model or it it's cool. It's like

their their new model where they've got

um Fable is kind of the parent and

they've got a sidekick.

Point is, you're not going to it's not

viable to run Fable or any super premium

model all the time, but have a a sort of

a parent, you know, very intelligent

model run children. You can do these

things, but in the end, you're going to

have to pay attention. Um, and it's just

like if you ran a big team and you were

out to lunch on your boat all the time,

you got to get in the office, and the

office is your phone. Um, and so get it

done. And so, Greg, this is my homework

for you. Next time we talk, I want you

to be doing more than 50% of your work

on your phone.

>> Yeah. I mean, that's why that's why

you're here, right? Like, you're a lot

of people ask me like, "How do you

choose guests?" And and why do you

explore topics, certain topics over

other topics? And it's it's really it's

for everyone. Like, I love arming people

with the information, the tactics, the

systems, but I'm also everyone. Like,

I'm a part of that, right? Like a part

of that is like I know that you're

you're doing this and I want to

understand why you're doing it. And I

also want to understand like you know

one of the takeaways I have is your

system for dealing with tasks is not is

not too dissimilar to a a very organized

person pre- AI right

>> like the idea of like pinning you know

your most important things like it's on

one hand it's pretty obvious but on the

other hand if no one tells you like

>> hey you know the way to to do this so

you stay sane is via that system you're

you it's overwhelming. Right.

>> Yeah, it is funny. It's like back to the

basics, right? It's just in an agent

age. Um, and it turns out like being a

very very good engineering manager is is

probably one of the most valuable skills

you can you can teach yourself. And you

know, I'm uh have the honor of sort of

mentoring uh my nephew right now and and

he's coming to me and asking me how do I

win? You know, he's just come out of

college soon. How do I win in this new

world? Right? And you know, I'm

basically teaching him what we're

talking about in the show. It's like,

okay, your job is going to be managing

as many agents as you possibly can. Get

good at that, right? Which means, you

know, you need to start to adopt these

these these behaviors, right? Work in

the cloud, prioritize what you're doing,

learn how to make high stakes decisions

quickly, get technical. Um, you know, it

it is a farce that engineering is going

away. Like if anybody tells you

engineers are going away or becoming

less technical, they're just not doing

the work, right?

>> Um and it's like being a carpenter. Like

say you had robot carpenters around. If

you were a real carpenter, you'd be a

much better manager of those robot

carpenters because you understand how to

make the cut. You know where to put the

nail. Um and so and so it's important.

So, to back up and if you're listening

to the show, like the thing I want you

to walk away is is cloud agents. Get

your reps in, figure out how that works.

Try it a couple times. Um, do it from

your phone and get comfortable, you

know, looking at your small phone

interface and and learning how to work

that way. Um, so I think that's that's

thing one. Uh, it's very important. The

second thing I want to dive into is

automations.

Okay, so say you're working in the

cloud. Well done. Um, you know, you're

starting to separate yourself from the

crowd. The second thing is automations.

Um,

you need to sort of think about things

that you would have normally had a

meeting with, uh, you know, once a week

with your team, uh, where you check on

something or daily you check on

something and build an automation. What

does that mean? Um,

so the first thing to do is go back to

your agent, start a session, and say,

"Okay, I want you to automate this task

every x amount of days." And so, one of

them that I do, it's really simple, is

that I want to automate the test of

signing up for Untangle, um, creating a

case, onboarding a client, having that

client go through discovery, you know,

basically our user experience. But I

want that to be automated in a browser.

Like this is not rocket science. This is

basic user testing, right? Um but it

turns out that good agents can do this.

And so the big unlock is uh set up your

agents so that they can make themselves

better um uh instead of you trying to

build these systems. Um and so I set

that up and then in Devon it's really

cool. It's called a playbook. But the

idea is it's it it's it's sort of like

here's how to do this thing. It's

different than a skill. It's more of a

list of things to do, how to do them

correctly. Um, and I have a uh it's

called endtoend

uh signup test and that runs three times

a week. Um, because it is expensive.

It's it's probably 60 bucks in tokens.

Um, because it's doing a lot of it's

doing a lot of browser testing. Um, and

uh, I think that's sort of a side point

I want to explain here is it's it's now

vital that your agent can can properly

brow browser test. Um, and it really

should work out of the box, right? This

is part of the reason I used Devon is

because they've been using cloud agents

that can do browser testing for 2 years

now. It's so good. It records a video.

It annotates the video. it it then looks

at its own video and it fixes bugs that

it sees, right? So you have this agentic

loop that is specifically around browser

testing that then is automated three

times a week, right? And then what what

what do you do with that? And then you

need to trigger a triage based off of

that. So say that it's like, oh wow,

during this the the third test failed in

this automation, I'm going to spin up a

Devon session or an agent session to

then fix it. Um, and then you have to

figure out where do you know that that

failed? How do you know? Did the agent

just kind of f, you know, silently fail

and then fix itself? What about the PR

that was created? So you have to start

thinking about how do you see these

things when they happen, right? as the

manager of all these agents, how do you

know uh what's failing, what's not? And

and that's again where you should talk

to your agent and say, "Okay, we've got

this end to end test. You know, I

understand it runs Monday, Wednesday,

Friday. Um I understand it, you know, it

it triggers a child triage session if

anything goes wrong. How do I know about

it?" And it and then your agent might

say, "Well, you know, why don't I post

in Slack?" You know, and and you can

say, "Well, okay, how are you going to

do that?" Well, we need to connect the

MCP. you know, how am I going to know to

look in that channel? There's just all

this machinery that you need to to to

automate and figure out and and work

through um to build that system um of

automation so that you are doing less of

the work.

>> So this specific automation is this more

of a like bug testing QA automation or

is it more of a um

you know UX QA perspective? Th this one

is more about um uh finding bugs that

have been introduced to prod that

somehow uh we missed in our in our user

tests um

>> in our automated test suite.

>> So that one in particular is like hey we

just can't have signups go down and not

know about it. Um, but you know,

sometimes your your automated test suite

just it it doesn't catch everything. Um,

but a good oldfashioned clicking through

the site in a browser catches that.

>> Um, it is not really about the UX. Um, I

think that there's there's no one thing

I will say is there's no substitute and

we probably won't have a substitute for

a while of you using your own app. there

just isn't a good substitute right now.

So, right now, every every so often, I

literally go through our entire app

myself. Um, and then I obviously quickly

spin up, you know, sessions to fix bugs

I find. But, um, we're we're just so far

from agents still. you the the smartest

models in the world still lack this

sense of obvious

you know uh intelligence where you're

like why did you think that was okay?

Well, yeah. I think the bigger question

is, you know, if if there's a division

of labor between human beings and

agents, what are the tasks that you as

the human being are going to be doing?

What are the sets of tasks? And what are

the sets of tasks that the agents are

going to be doing that you're okay with

them doing and keeping you in the loop?

So, my question for you around

automations is beyond beyond QA, what

are some other automations that you know

a founder could be implementing? So um

let me pull up my list for you. So the

kind of automations that I have are uh

clearly uh uh QA bug fixine. Um the

other is what I call my production

watchdog. Um so we've got a lot of real

law firms using Untangle. got a lot of

real clients, you know, going through

discovery and we've got just a huge

amount of activity and and honestly I

can't keep track of it uh you know

mentally by just looking through logs.

And so what I do is every day at 9:00

a.m. I have what's called a production

watchdog automation which what it does

is it goes through all of the events in

the database um that happened for our

paid customers and it summarizes that in

uh a JSON file and then that JSON file

is in our admin and then I I you know

know every morning I go in and I click

on uh the production watchdog and I read

through the summary of what the

customers did um and it's so valuable.

Um, so it's sort of like you can

imagine, you know, your chief of staff

showing up and saying, you know, all

right, Greg, here's the important stuff

that happened yesterday for our

customers, right? Um, and I'm going to

roll it up and explain, you know, the

things that went well and a couple bugs

that we saw. Um, so that's an

automation. Um, essentially what

happened yesterday that was important.

Um, so that one

>> that's a big one.

>> Yeah,

>> that's honestly a big one. like it's

it's like one of those ones that sounds

small but is a big one because you're

going you think you know what's going on

but you don't really

>> you don't I mean and it's it's so

shocking to me like yeah the amount of

times I'm like oh uh and this is the

other trick man so use a production

watchdog automation pick your most

important customers and then make sure

that in that report it's like the

customer did X here's a link to view

that right and so I actually have a

production instance where you click it

and you actually see what the customer

was doing and then I'll be like whoa

that's weird like and we have this

amazing ability that you know over you

know uh you know millions of years of

evolution in very quickly picking up

things that are off. [laughter]

And so there's a couple times I'm like,

"That's weird." And I click into it and

it turns out, oh, the user experience um

that the agent built on that wasn't was

wasn't quite what I thought it was. Um

and so yeah, production watchdog

automation that links out to real UI um

is just man, it's a game changer. So

that's a big automation. Um this is a

really important a self-improvement

loop.

How does this work? Okay, everyone's

talking about self-improvement, right?

>> Um, so in Untangle, we have an agent

called Grace, and she is essentially a

parallegal. Um, and she has a lot of

chats with both uh our customers who are

attorneys, our customers who are

parallegals, and our customers and their

customers who are clients that are

getting divorced. Um, so how do I

actually grade those chats and then

improve them uh without managing the

fine details? So what I do is I every

day I have an automation that looks at

these chats and then grades them on a

rubric. And so you and again just talk

to your agent about this. You pick the

most important part of your app that you

want to self-improve and say here's how

you agent judge whether this thing that

happens is good or bad. Um and then

every day I want this automation to kick

off. Look at those things for us.

They're conversations that Grace is

having. and then I want you to grade

them. And then if anything is bad or

below this score, I want you to kick off

a child session and fix it. And it is

shocking to me the number of small, you

know, fine UX details that have been

picked up by this loop and have been

fixed. And honestly, there are things

that I just wouldn't bother doing.

Either I didn't know about them or I'd

be like, me, you know, it's a paper cut.

Like, are we really going to, you know,

ship a fix to that? But because the

agent already identified it, spun up a

PR, and it's ready to ship, I just go

ship it. [laughter]

And you know, we probably ship I would

say probably three of these a day.

>> Wow.

>> That's that's huge.

>> And you know, a lot of a lot of the

backlash on on loops is is just that

it's expensive. Um, how how how have you

with with Devon specifically, has it

been like absurdly expensive? Like what

are we talking? No. So he Well, so

here's the thing. So last month I spent

probably 20 grand in in tokens, uh,

which is just too much. Like it's not

viable. I I think all of us are in a

place where we we're getting to the spot

where it's realizing, okay, for real

engineering work per employee, you're

looking at probably 5 grand a month.

Like that's probably where we're going

to shake out here. And anything more

than that, you you really need to figure

out model routing. Um and so thankfully

Cognition knows this and every

everyone's figuring this out like yeah

whether it's cursor cognition

>> I would say factory AMP all of the

independent agent labs understand like

[laughter] we our customers have a limit

like they're not going to pay infinite

token budgets

and then you have the this is me ranting

a bit but it's like you have cloud code

and codeex who are just severely you

know fund they're they're funding tokens

and making it seem cheap, but this is

not going to work long term. And so what

I do is basically I have learned how to

use the right model for the right task.

And so these reinforcement loops, they

use what's called SUI 1.7. It's a

fine-tuned model that Cognition has

specifically built for coding. It's

super cheap um compared to, you know,

Opus 48, you know, GPT56. And so a lot

of these loops happen with this uh you

know cheaper you know finely tuned model

and we're just going to see more of

that. So you're talking I don't know it

you know actually I could probably I

could probably look and dig it out but

you're t probably talking like five

bucks a session on that.

>> Okay that's not bad.

>> I mean and if you're not willing to pay

15 bucks a day

>> to improve one of the core you know

feature sets of your like what are you

doing? Like either it's not a real

company or you don't care.

>> Yeah.

>> Totally. spend the money like and and

then you know figure out the the system

uh to reduce your token cost. I will say

this is why I don't understand why

anybody would build their engineering on

a frontier lab stack. Like I'm just

going to call it out like if you are if

your whole engineering uh you know

motion is happening inside of cloud code

or inside of codeex what are you doing

like because they are not incentivized

to make it reasonable for you long term

right they're going to lock you in to

their models to their process whereas if

you use an independent agent lab you

know like an AMP like a Devon like a

factory like sort of a cursor but

they're kind of weird now those systems

are going to optimize for affordable

engineering long term. Um, and uh, I

don't think you want to be locked in to

anthropic or or open AI, you know,

solely.

>> I mean, to be clear, you can still use

their models within some of these

products, right? So, you're not

abandoning ship.

>> No, it's actually so it's it's yes and

right. So the beautiful thing is if you

pay you know an independent agent lab

like a factory like a cursor like a

cognition like an amp you know etc

they're incentivized to figure out how

do I give you the best results for the

lowest price. So they'll they'll say,

"Okay, we're going to use GPT56 soul on

extra high for this type of task, and

then we're going to route, you know, to

a uh you know, to a SWE 1.7, you know,

and then we're going to have it checked

with a Fable." Like, they're going to

figure out that hard stuff.

>> Um, and the other thing I will say is if

you're trying to build a software

factory for yourself, stop. like it, you

know, the reason why you see, you know,

Ramp launch, Inspect, which is their

custom in in-house agent, is because

they got to a size where they had to

build their their software factory

in-house. And if you get to a certain

size, you will do that. But now, if

you're a oneperson shop and you're just

building something that's just you, you

can probably get away with this $200 a

month uh, you know, angle on a pick your

frontier lab. But as soon as you

graduate into, okay, we're building a

real product. We have product market

fit. We're going to start hiring people.

You got to build a software factory. And

at that point, you want to be on an

independent agent lab. Like otherwise,

you're going to have to build it

yourself, which is stupid, or you're

going to pay and be locked into, you

know, a walled garden. Um, and you can't

do that. The the analogy I use for it

is, you know, working with an

independent lab is sort of like a uh

like working with like a mortgage broker

or travel agent. Like they're going to

make a lot of calls to make sure to

[laughter] get you the best price and

stuff like that. Um

>> that's an Yeah, to me I use both. You

know, I I'm in cloud code, but I'm also

I use cursor personally. Um, although

Devon I'm interested in I might if

people are interested I might just do

like a whole

breakdown on Devon if people if people

want to want to do that.

>> Devon's so good now. Um, you know

>> I just it's so good. It it's amazing.

Um, but I will say I do use codecs,

right? The the the the Mac app is so

beautiful like

>> and there's and they are subsidizing the

tokens so much that if you want an agent

on your machine just to do really cool

helpful stuff, open browser tabs and

create Google Docs and check your email

and and do all this information work and

and basically get it for free. I mean,

you're paying 200 bucks and you get

almost infinite tokens. like awesome,

but don't build your company software

factory on that. Like it I just think

it's bonkers to do that.

>> Mhm. We've covered a lot today. Is there

anything else you wanted to uh

[laughter] cover?

>> I feel like I'm just vomiting, you know,

all all the stuff in my head that

>> Well, no, I it's important that like

this is what I wanted. I wanted you just

to vomit everywhere because like it's

sort of like advice that you you you

would give to your to your nephew I

think um or 22-year-old um

>> you know a 22-year-old fresh out of

college. It's like what are the minds I

need to know in terms of

>> managing agents? So is there anything

else that people need to know? Cloud

agents, automations.

Yeah, cloud

automations like th those are the two

that I want people to walk away from. I

guess the third is is really you will be

using a software factory like and what I

mean is the agents are going to be

writing 100% of your code, reviewing

100% of your code, shipping 100% of your

code. Like that's where we're going. Um

and so the the more you can get there,

the the better. thing one uh thing two I

think you know thing three is um it it

is important to build um uh a credible

reputation for yourself right and I will

say you know the work I put in over you

know the last 20 years on X like it

really pays dividends and what I mean by

that is if you if you share what you're

learning publicly and you generally give

like write very helpful articles on X.

Um, and obviously the ALGO really does

reward articles still. So, you should be

writing articles on X. Um, you should

absolutely be on X, no matter how you

feel about Elon. Um, uh, because sharing

your knowledge really does build up

credibility, which will pay dividends.

And I don't mean just like getting paid

to post. I mean the the the the network

and the relationships. I mean, the

reason why I'm on the show is because we

built a relationship over X. Um, and so

I would encourage people take the time

to share what you're learning. It

doesn't have to be super polished. Um,

but get out there and be useful to

people. Um, and it will open up doors.

Um, so, uh, you know, more than you

think. Just get out there and share it.

Um, and, uh, and you'll reap the

rewards. Not immediately, but

eventually.

>> I mean, even if you don't know, you can

just say, "I don't know." And that's

your and that's your strategy, right?

And that's interesting, too.

>> Amen. And if you don't know, then ask an

agent to help you.

>> Yeah. So,

>> I mean, my original co-host of this show

was Sahel Bloom. Sahil Bloom worked in

private equity and during COVID he just

started writing about topics he was

interested in like he would be like just

finance topics or you know particular

individual who was interesting and it

like it it it read as like Wikipedia

articles but really focused on Twitter

specifically like optimized for Twitter

and and that was his thing and he wasn't

saying I know everything he was just

saying like I'm learning this thing.

Here's the thing I'm learning. He grew

his audience over a million Twitter

followers, wrote a bestselling New York

Times bestselling book, started

companies on top of it, raised a fund on

top of it. So, anything is possible for

sure. Uh on on X.

>> Yep. Just just get out there and do do

and learn. Um

>> uh you know, there there's no try.

There's only do as Yoda says, right? So,

>> totally. Uh Ryan Carson, I feel smarter

and that's that was my goal. Um

and uh speaking of X, you're a wonderful

follow. I'll include links where you can

follow Ryan across the internet,

including X. Um he's a mustf follow, so

please do that. Thank you for coming on,

sharing your generous uh thoughts,

incredible setup, and uh is there

anything you want to leave people with?

No, thanks Greg. Thanks for having the

show. I appreciate you putting out the

constant knowled uh constant knowledge

and sharing. Um it really makes the

internet a better place. So happy to be

here and thanks for having me.

>> All right, I'll see you next time.

Take care.