So, you've been hearing about this

Claude Code lately, or maybe you even

went out and got an Anthropic

subscription for 20 bucks, and you're

using Claude Code a little bit, and you

wonder, should I pay 200 bucks? Is it

worth it? Today, we are definitely going

to find out. Oh man, did the vibes go

bad? Oh, good gosh, how far did these

things get? This is amazing. Holy

mackerel.

[Laughter]

All right. What What is going on,

Claude? I'm going to throw the same

request, the same PRD at both Opus and

Sonnet. That's the differentiation

between the $20 plan and either one of

the Max plans. And we're going to see if

there's an actual difference between the

output on these and whether or not it's

immediately needed that you upgrade from

20 bucks to $200 or maybe you don't need

to or exactly when you should. Those are

all the questions that we're going to

look into today. Let's dive in. This is

just kind of an experimentation project

today. Let's go have some fun. Okay. I

love these kinds of things. I hope you

do, too. This is just me running some

experiments in two different modes. So

on the left we have Opus and on the

right we have Sonnet. I have prefilled

the project a little bit. So I've got

Nex.js installed and shad CN just so

that they could be kicked off

appropriately. What I've done is I've

worked with 03 in chatbt to try to get a

very explicit PRD which is a

requirements document. I also have a

design doc which gives a very brief idea

of what we're trying to build. I will go

into each one of these and make sure

that this one is set to opus and this

one is set to sonnet so that we lock

them down. And lastly, what I will show

you is the design dock themselves. So

what I've done is I've broken it into

two parts. We'll do phase one and it

will hopefully build a bit of the

architecture and then the second one

over here. It's got the same top, but

then it has the intelligent variance.

So, how do we use LLMs and that kind of

information to enhance what we're

building? And so, I wanted to say, okay,

if I only had the $20 plan, what kind of

outcome do I have with the same starting

point? If I have an expensive plan, what

kind of outcome do I have? That's

basically what we're running. Frankly, I

did this yesterday and both of them fell

on their face. So, I gave these two

phases that you just saw me describe. I

gave them both at the same time and I

didn't give any of the technical

details. I let them do the planning. I

thought well if you do have one or the

other maybe the technical plan is also

something that we want to evaluate and

the problem was they both went in kind

of very different directions and fell on

their face for different reasons and it

became really difficult to evaluate.

Okay, if we look at the anthropic docs,

really what I want to point out is just

a few simple things is they say Claude

for Opus delivers sustained performance

on longrunning tasks that require

focused efforts and thousands of steps

with the ability to work continuously

for several hours and it dramatically uh

outperforms all the other sonnet models.

Okay, I got them both started and then I

pulled up the browser. So remembering

this is Opus over here. Opus does run

excellent. So, this already looks pretty

good. On this side, I have a single

error. I am not going to fault anything

for having a single error. So, let's

collapse that and give it that error and

give it a chance to fix it. If I was

asking whether or not I needed to spend

10 times more money to get a

subscription so that I could do the same

kind of work, I'm willing to fix one

bug. And it turns out I fixed I just

pasted that one bug in and it did fix

it. At least it got it to this point.

Now, admittedly, we don't think this

project is done yet. So, I don't want to

go too far, but this is fantastic. Just

seeing this. By the way, like I said, I

did this yesterday and oh man, did the

vibes go bad. All right, so I'm going to

stop here and say, let's move on to

phase two just to say this is where it

got us for phase one. What does phase

two look like? I will peek under the

hood of these. Here we go. I'm going to

let you help me peek under the hood. I

will not judge them for any of this

being wrong. So, we're going to create a

project called librarian. That one

worked. Let's create a project called

librarian. Okay. All I am so gunshy from

yesterday. This is already great. So,

let's open this project first. All

right. Looking pretty pretty reasonable.

Open this project. I mean, these are

great. Do I have to keep going? I'm just

nervous. I know you're not nervous with

me, but All right. Ah, that looks

brilliant.

Also brilliant. Oh, good gosh. How far

did these things get? This is amazing.

Uhhuh. All right, let me resize these so

that we can see. Yeah, these are nodes

that you'll be able to move around on a

board as expected. Holy mackerel.

Okay, this is so many steps ahead of

what I hit yesterday. This is where I

thought I would get. So, I'm hugely

excited here. I'm going to stop here

because I do know that it has not tied

in some of the other moving parts. I

mean, do I know that? Okay, let's let's

put something in here. Okay. All right.

I have not gone any further than this.

I've just filled in uh small elderly

female librarian and a little piece of

text. And first, let's hit create clips.

I don't expect it to do anything, but

let's give them a shot. All

[Laughter]

right. What What is going on, Claude? Oh

my god. Is anyone else freaking out

right now? I did not expect it to get

nearly this far. All right. The designs

are definitely different, but they're

both frankly quite attractive. All

right. Are we ready? This one's 8

seconds long, apparently. Oh, it doesn't

play. Okay, that's fair. Doesn't play.

Not shocking. And we definitely have an

issue. No supported sources. Fine with

it. Fine with it. This is so much

further than I expected. Let's try on

this side. Excellent. Same thing. Okay.

So, this got us incredibly far. I don't

have the variance. That's actually in

phase two of the document. So, let's go

add that so that we can see what the

variants work like. But I don't know if

you could tell. I'm actually blown away

by this. Oh, I can't wait. I got to go.

Here we go. Okay. I'm gonna cover really

briefly what's inside of this thing.

This thing is I'm It's relatively

sophisticated, frankly. It's not one

moving part that's in here. We've seen

some of this already with that

mindblowing first pass of the PRD. It

created an endless canvas. There are

utilities out there that it's using. So,

it did not build all of that. It's using

open-source solutions. They're both

using the same intentionally. That was

part of my brief. They both put that

inless canvas down perfectly. There is a

set of LLMs that we'll be using and

that's the variance. We'll see that in

the next step. This is this step of I

want to create I have a voice. So

there's just a small elderly librarian,

but then I want to be able to use

intelligence to creatively alter the

prompt that I'm putting in. That's not a

great prompt obviously. So, we would

want something that's more descriptive

and kind of written in a way that a

voice model, which is we're using 11

Labs. You'll see this API down here is

an API that allows us to do text to

speech. And this in this case, what

we're allowed to do is create unique

voices and then do text to speech. So,

we're really trying to create new

characters with this definition. And

then there's a document database that's

storing all of this stuff so that we can

keep it in projects, come back to it.

So, that's the build. This is not a

small set of components to build from

just a simple PRD. So, I'm excited we

got this far. It's time for me to put

the second prompt in. Okay, I've put the

second prompt in. Let's let this cook

for just a minute. See what I can come

up with. Uh, they both finished again,

which is great. I want to try the create

variants. Let's create a variant off of

that. And here's another character. All

right. small elderly female librarian

but with exhausted night shift tone but

with heavy French accent. Okay. Uh all

right. So this is just very repetitive.

Something's going on. This feels

algorithmic. Okay. So full disclosure,

there's been a lot of back and forth.

There's a couple things that were

findings out of all of this. Really

still a pretty good experience and like

we mentioned kind of a sophisticated

project, but they're both ready. Let's

go take a look at what they have for us.

All right, here we are. We have Opus on

the left. We're going to create a new

project. Call it the cowboy. We'll do

the same over here. All right, let's

move into the cowboy in both cases. As

before, excellent looking. So, here we

are with our initial nodes for each one

of these. So, let me grab the data that

we're going to need to put into these.

So, what we have, if you recall, we can

create variants, which is basically a

new character. This one, in fact, called

it create characters. And if you hit

create characters, what it's going to do

is it's going to reach out and talk to

two LLMs as we talked about before and

create a new set of characters that are

related. What we've got is a warm,

easygoing male voice with a slight

southern draw on a calm, steady rhythm,

mid-range pitch with a touch of gravel

like someone who spent years outdoors.

So, it's kind of a cowboy sound at this

point. Um, so I will look into some of

these characterizations as well. And

here you'll see this warm, easygoing

male voice retains its slight southern

draw and calm, steady rhythm, but now

carries a pronounced Appalachian twang.

Okay, that one's interesting. Easygoing

male, southern draw, calm, steady

rhythm. Sounds like a man who's seen

some things, right? So, these aren't

terribly unique. I'm not so sure these

are really uh that creative yet. So,

that prompt probably needs some work.

All right. So, let's create some clips

for the grandfather and create some

clips for our original cowboy over here.

So, let's start with, "Well, now reckon

life's like riding a wild horse. You

don't get to choose the trail, but you

sure as hell decide how long you stay in

the saddle."

>> Well, now reckon life's like riding a

wild horse.

>> Okay, that one's interesting. I really

like this one quite a bit, actually. I

could use that one. This one sounds like

a cowboy on a telephone telling a story

from the old days, basically. That one's

kind of interesting. All right. What

does this guy over here got?

>> Well, you know, when I was just a young

fella, we used to spend every summer out

by the old oak tree, watching the stars

light up the night sky.

Well,

you know, when I was just a young fella,

we used to spend every summer out by the

old oak tree watching the stars light up

the night sky.

>> Okay, I like that one. So, this is

another one that I like quite a bit. And

of course, then you would just start

working from here. So, we could take

this one and say space cowboy,

but not just space cowboy. Let's create

a new one from him and start creating

other variations of a space cowboy.

Rugged space cowboy.

>> Well, now, partner, ain't it a fine

sight to see the

>> No, no, no, no. This is too boring. Out

here among the stars, you learn real

quick that justice ain't just a word.

It's a way of standing tall.

>> Out here among the stars, you learn real

quick that justice ain't just a word.

out here among the stars. You

>> No, no, no, no, no, no. Okay, so let's

see. We got kind of a crazy figure now.

Let's see if that helps.

>> Keep your heads low and your mouth shut.

You move when I say not a second before.

These bots don't miss. And I'm not

dragging any more kids back in pieces.

Let's go.

>> Awesome. That was really good. Let's try

the ones from Sonnet. It doesn't really

matter. It's all 11 Labs at this point,

but it's fun. Keep your heads low and

your mouth shut. You move when I say not

a second before. These bots don't miss.

And I'm not dragging any more kids back

in pieces. Let's go.

>> All right. I mean, honestly, that's a

success. So, there are some nuances

here. You will still need to engineer

things. So, I don't want to paint too

rosy a picture that you can just create

this cold from a few prompts. But

really, honestly, it got super close to

creating this really well from just a

few prompts. And then the engineering

part of it kind of really takes hold. Uh

so I would call this frankly a complete

success on both of these. Okay, closure

here. Right. So this one I'm struggling

with just a little bit. It was a great

experiment to run these side by side.

Oh, hold up. One last thing. You just

thought we were done. Let's go take a

look at this special surprise that I've

got for you. Yeah. Yeah, that's right.

You knew I was going to do it. You knew

it was happening behind the scenes. I

can't fool you anymore. I decided to

also do this in Gemini. And I will

admit, I did not go nearly as far with

Gemini. It got to a success place very

quickly, but I never got to the

completion of pushing back in multiple

times to create the files and a couple

other things. Let's take a look. This is

the project as Gemini created it. And it

it really this is many times I've done

this with Gemini as a comparison between

this and Claude Code. And Gemini's

design is never as thoughtful. It's very

fast. It's probably twice as fast,

frankly. And its engineering seems

really pretty good, but it just does not

create any thoughtful design whatsoever.

So, if we go into the librarian project

here, you can see like it won't save the

locations of things. I didn't work my

way through that, but you can see many

of the other parts. So, it can create

clips even though they won't play. I

didn't go back and say use the the file

as a clip. Trying to get to one. Okay.

Well, let's use this character. If we

create a clip, it'll go and create

clips. If we create variants, it'll go

down and create more variants. So, this

was Gemini completing very quickly and

truly for free. I mean, I can't say

enough about free. If you were looking

for an outcome of this, I know you

weren't expecting Gemini CLI again. And

I love 2.5. This is the 25 Pro model.

So, I really do like what it does, but

honest to gosh, it does not compare to

the completion aspect that Gemini is

bringing to things. And really, the

design here to some degree matters. Some

of the thoughtfulness of the way the

error messages and things like that

happen are really pretty well thought

out. even the layout of the screen and

what it looks like to start the

application. These are pretty obvious

things for it just to create a label at

the top and a linked list below it is

super old school. So it feels like it's

meeting the spec rather than kind of

using some intelligence to figure out

how it can meet the spec creatively. All

right, so that's my Gemini comparison. I

wanted to share that. You knew it was

happening behind the scenes. Success.

Okay. And now for the actual uh kind of

conclusion here. What do I think? Is it

better? Like full stop. No, it is not

better period. Was it better during

development to have Opus? No. It took

longer. Sonnet did a fantastic job.

Finished much faster than Opus and had

far fewer errors, frankly, that it

couldn't get out of. In fact, I didn't

run into any that it just couldn't get

out of. And I think in there is the

surprising answer. So agentically, I

think all models become far more

performant with a good solid agentic

loop. Anything that has an evaluation

process and is really thoughtfully

utilizing the tools that it has is going

to be far more performant than the model

itself. And I think that's the answer.

What's happening here is claude code is

the agent loop. What's happening in

Claude Code I believe is partially

inside of Opus is this agentic kind of

comparative evaluator that's going on

inside of that thinking loop is what

Claude code has built into it so that it

can use something simpler like sonnet

and so sonnet starts to behave a lot

more like opus and I think that's why a

lot of people that are using this are

starting to say I use sonnet a lot I do

go into opus when it's a big problem

even I have used opus in a scenario

where I thought wait this is this is not

solving the problem. Sonnet is going in

circles and having the same problem. Let

me go up to Opus and it really has

solved the problem. So, there is a value

to it. It's there. But is it worth $80?

I would say no. Is it worth you know

$180?

Definitely. Unquestionably no. However,

the one thing and this is this is the

final takeaway. Best thing I can tell

you is I think Anthropic has their

metering done just right. The way that

they limit it per five hour block allows

you to bump into your thresholds at a

given moment and realize you're getting

closer to it and understand that oh my

gosh in a couple hours, 2 hours, 3

hours, I'm going to have to refresh. I

have to wait until that time. So, I

either slow down now or I'm kind of out

until that time rather than I'm done

until the next day or in case in in many

cases V3, I'm looking straight at you.

Uh, you're done for the month.

Congratulations, you're on day three.

You got no more credits. You're out of

here. That really sucks. So, with that

awesome metering that they have, you'll

know when you need to upgrade. This is

the beautiful part. I think Anthropic

built exactly the right set of tools. I

mean, we I cannot applaud enough for

what they've built. Go get the $20. Use

Claude code because it is literally

fantastic. $20 absolutely will get you

there until you notice that you're being

limited. Once you're being limited, you

realize, oh, I'm going to have to move

up because I use too much and that's it.

And if you start hitting that limit when

you're on the $100 plan, you got to move

up again. So, they've really done it

exactly right. And it's not really the

Opus upsell. The upsell is excellent.

It's upsell so that you can get, you

know, a little bit more traction on your

coding efforts. That's it. That's the

takeaway. I know. I think we were all

hoping that Sonnet would clobber Opus in

some crazy way and we'd all go, "Oh, we

don't need the $200 model." And you

really don't. My answer, I guess, is you

really don't unless you need it, but

you'll know you need it. It's It's not a

surprise. Don't Don't FOMO over it. Get

the $20 plan. You'll be perfectly fine.

Okay. I hope you got something out of

this one. Really, thanks for coming

along for the ride on this one and I'll

see you in the next