All right, this is the episode where  my roommate teaches me semiconductors. 

It's also the send off for this current set. It is. After you use it, I'm like,  

"I can't use this again. I gotta get out of here." 

No sloppy seconds for Dwarkesh. Dylan is the CEO of SemiAnalysis. 

Dylan, here’s the burning question I have for you. If you add up the big four—Amazon, Meta, Google,  

Microsoft—their combined forecasted CapEx this  year that you published recently is $600 billion. 

Given yearly prices of renting that compute,  that would be close to 50 gigawatts. 

Obviously, we're not putting  on 50 gigawatts this year,  

so presumably that's paying for compute that is  going to be coming online over the coming years. 

How should we think about the timeline around  when that CapEx comes online? Similar question  

for the labs. OpenAI just announced they  raised $110 billion, and Anthropic just  

announced they raised $30 billion. If you look at the compute they  

have coming online this year—you should  tell me how much it is, but is it on the  

order of another four gigawatts total? The cost to rent the compute that OpenAI  

and Anthropic will have this year to sustain their  compute spend is $10 to $13 billion a gigawatt. 

Those individual raises alone are enough  to cover their compute spend for the year. 

And this is not even including the revenue  that they're going to earn this year. 

So help me understand: first,  what is the timescale at which  

the Big Tech CapEx actually comes online? And second, what are the labs raising all  

this money for if the yearly price of a  one-gigawatt data center is $13 billion? 

So when you talk about the CapEx of these  hyperscalers being on the order of $600 billion,  

and you look across the rest of the supply chain,  it gets you to the order of a trillion dollars. 

A portion of this is immediately for compute  going online this year: the chips and the  

other parts of CapEx that get paid this year. But there's a lot of setup CapEx as well. 

When we're talking about 20 gigawatts of  incremental added capacity this year in America,  

a portion of this is not spent this year. A portion of that CapEx was actually  

spent the prior year. When you look at Google  

having $180 billion, a big chunk of that is  spent on turbine deposits for '28 and '29. 

A chunk of that is spent on data  center construction for '27. 

A chunk of that is spent on power purchasing  agreements, down payments, and all these other  

things they're doing further out into the future  so they can set up this super fast scaling. 

This applies to all the hyperscalers  and other people in the supply chain. 

So with roughly 20 gigawatts deployed this year,  a big chunk is hyperscalers, and a chunk is not. 

For all of these companies, their biggest  customers are Anthropic and OpenAI. 

Anthropic and OpenAI are at roughly  two to two-and-a-half gigawatts right  

now, and they're trying to scale much larger. If you look at what Anthropic has done over the  

last few months, with $4 billion  or $6 billion in revenue added,  

we can just draw a straight line and say they'll  add another $6 billion of revenue a month. 

People would argue that’s bearish,  and that they should go faster. 

What that implies is they're going to add $60  billion of revenue across the next ten months. 

At the current gross margins Anthropic had,  as last reported by media, that would imply  

they have roughly $40 billion of compute spend for  that inference, for that $60 billion of revenue. 

That $40 billion of compute, at roughly  $10 billion a gigawatt in rental costs,  

means they need to add four gigawatts of  inference capacity just to grow revenue. 

That’s assuming their research and  development training fleet stays flat. 

In a sense, Anthropic needs to get to well  above five gigawatts by the end of this year. 

It's going to be really tough for  them to get there, but it's possible. 

Can I ask a question about that? If Anthropic was not on track to have  

five gigawatts by the end of this year, but it  needs that to serve both the revenue that's gone  

crazier than expected—and maybe it's going to be  even more than that—plus the research and training  

to make sure its models are good enough for next  year: Where is that capacity going to come from? 

Dario, when he was on your  podcast, was very conservative. 

He said, "I'm not going to go crazy on  compute because if my revenue inflects  

at a different rate, at a different  point… I don't want to go bankrupt. 

I want to make sure that we're being  responsible with this scaling." 

But in reality, he's screwed the pooch  compared to OpenAI, whose approach was,  

"Let's just sign these crazy fucking deals." OpenAI has got way more access to compute  

than Anthropic by the end of the year. What does Anthropic have to do to get the compute? 

They have to go to lower-quality providers  that they would not have gone to before. 

Anthropic historically had the best  quality providers, like Google and  

Amazon, the biggest companies in the world. Now Microsoft is expanding across the supply  

chain, and they're going to other newer players. OpenAI has been a bit more  

aggressive on going to many players. Yes, they have tons of capacity from Microsoft,  

Google, and Amazon, but they also  have tons with CoreWeave and Oracle. 

They've gone to random companies, or companies  one would think are random, like SoftBank Energy,  

who has never built a data center in their life  but is building data centers now for OpenAI. 

They've gone to many others,  like NScale, to get capacity. 

There's this conundrum for Anthropic because  they were so conservative on compute,  

because they didn't want to go crazy. In some sense, a lot of the financial  

freakouts in the second half of last year  were because, "OpenAI signed all these  

deals but they didn't have the money to pay  for them…" Okay, Oracle's stock is going to  

tank, CoreWeave's stock is going to tank. All these companies' stocks tanked,  

and credit markets went crazy because people  thought the end buyer couldn't pay for this. 

Now it's like, "Oh wait,  they raised a ton of money. 

Okay, fine, they can pay for it." Anthropic was a lot more conservative. 

They were like, "We'll sign  contracts, but we'll be principled. 

We'll purposely undershoot what we think we  can possibly do and be conservative because  

we don't want to potentially go bankrupt." The thing I want to understand is, what does  

it mean to have to acquire compute in a pinch? Is it that you have to go with neoclouds? Do they  

have worse compute? In what way is it worse? Did you have to pay gross margins to a cloud  

provider that you wouldn't have otherwise had to  pay because they're coming in at the last minute? 

Who built the spare capacity such  that it's available for Anthropic  

and OpenAI to get last minute? What is the concrete advantage  

that OpenAI has gotten if they end up  at similar compute numbers by 2027? 

Are they just going to end this  year with different gigawatts? 

If so, how many gigawatts are Anthropic and  OpenAI going to have by the end of this year? 

To acquire excess compute, yes,  there is capacity at hyperscalers. 

Not all contracts for compute  are long-term, five-year deals. 

There's compute from 2023 or 2024, or H100s  from 2025, that were signed at shorter terms. 

The vast majority of OpenAI's compute is signed  on five-year deals, but there were many other  

customers that had one-year, two-year,  three-year, or six-month deals, on demand. 

As these contracts roll off,  who is the participant in the  

market most willing to pay price? In this sense, we've seen H100 prices  

inflect a lot and go up. People are willing to  

sign long-term deals for above $2 even. I've seen deals where certain AI labs—I'm being a  

little bit vague here for a reason—have signed at  as high as $2.40 for two to three years for H100s. 

If you think about the margin, it costs  $1.40 to build Hopper, across five years. 

Now, two years in, you're signing deals for two  to three years at $2.40? Those margins are way  

higher. Now you can crowd out all of these other  suppliers, whether Amazon had these, or CoreWeave,  

or Together AI, or Nebius, or whoever it is. These neoclouds are the firms that had a  

higher percentage of Hopper in general  because they were more aggressive on it. 

They also tended to sign shorter-term  deals, not CoreWeave but the others. 

So if I want Hopper, there  is some capacity out there. 

Also, while most of the capacity at an Oracle  or a CoreWeave is signed for a long-term deal  

in terms of Blackwell, anything that's  going online this quarter is already sold. 

In some cases, they're not even hitting all the  numbers they promised they would sell because  

there are some data center delays, not just those  two, but Nebius, Microsoft, Amazon, and Google. 

But there are a lot of neoclouds, as well as some  of the hyperscalers, who have capacity they're  

building that they haven't sold yet, or capacity  they were going to allocate to some internal use  

that is not necessarily super AGI-focused,  that they may now turn around and sell. 

Or in the case of Anthropic, they don't  have to have all the compute directly. 

Amazon can have the compute and serve Bedrock,  or Google can have the compute and serve Vertex,  

or Microsoft can have the compute  and serve Foundry, and then do a  

revenue share with Anthropic, or vice versa. Basically, you're saying Anthropic is having to  

pay either this 50% markup in the sense of the  revenue share, or in the sense of last-minute  

spot compute that they wouldn't have otherwise  had to pay had they bought the compute early. 

Right, there's a trade-off  there. But at the same time,  

for a solid four months, everyone was saying to  OpenAI, "We're not going to sign deals with you." 

That sounds crazy, but it was  because, "you don’t have the money." 

Now everyone's saying, "OpenAI,  we believed you the whole time. 

We can sign any deal because  you've raised all this money." 

Anthropic is constrained in that sense. There are not that many incremental buyers of  

compute yet, because Anthropic hit the capability  tier first where their revenue is mooning. 

That's interesting. Otherwise you  might think having the best model is an  

extremely depreciating asset, because three  months later you don't have the best model. 

But the reason it's important is that you  can sign these deals, lock in the compute  

in advance, and get better prices. Maybe this is an obvious point. 

But at least until recently, people had made this  huge point about the depreciation cycle of a GPU. 

The bears, the Michael Burrys or  whoever, have said, "Look, people  

are saying four or five years for these GPUs. Maybe it's because the technology is improving  

so fast, but it in fact makes sense to have  two-year depreciation cycles for these GPUs,"  

which increases the reported amortized CapEx  in a given year and makes it financially  

less lucrative to build all these clouds. But in fact you’re pointing out that maybe the  

depreciation cycle is even longer than five years. If we're using Hoppers—especially if AI really  

takes off and in 2030 we’re saying, "We have  to get the seven-nanometer fabs up, we have  

to go back and turn on the A100s again"—then the  depreciation cycle is actually incredibly long. 

I feel like that's an interesting financial  implication of what you're saying. 

There's a few strings to pull on there. One is, what happens to depreciation of GPUs? 

I guess I didn't answer your prior question,  which is that I think Anthropic will be able to  

get to five gigawatts-ish, maybe a little  bit more by the end of the year through  

themselves as well as their product being  served through Bedrock, Vertex, or Foundry. 

I think they'll be able to get to five or six  gigawatts, which is way above their initial plans. 

OpenAI will be roughly the same, actually  a little bit higher based on our numbers. 

But anyway, the depreciation cycle of a GPU. Michael Burry was saying it's three years  

or less. That’s sort of his argument.  There are two lenses to look at this. 

Mechanically, there's a TCO model, total cost of  ownership of a GPU, where we project pricing out  

for GPUs and build up the total cost of a cluster. There are a number of costs: your data center  

cost, your networking cost, your smart hands and  people in the data center swapping stuff out. 

There's your spare parts, your  actual chip cost, your server cost. 

All these various costs get lumped together. There's some depreciation cycles on it,  

certain credit costs on it. You build up to, "Hey, an H100 costs  

$1.40/hour to deploy at volume across five  years if your depreciation is five years." 

If you sign a deal at $2/hour for those five  years, your gross margin is roughly 35%. 

It's a little bit above that. If you sign it for $1.90, it's 35% roughly. 

Then you assume at that fifth year,  the GPU falls off a bus and is dead. 

In some cases, the argument people are making is  if you didn't sign a long-term deal, because every  

two years NVIDIA is tripling or quadrupling the  performance while only 2X-ing or 50% increasing  

the price… Then the price of an H100… Sure  maybe the value in the market was $2 at 35%  

gross margins in 2024, but in 2026, when Blackwell  is in super high volume and deploying millions  

a year, you’re actually now worth $1/hour. And when Rubin in '27 is in super high volume—even  

though it starts shipping this year, it’s super  high volume next year—doing millions of chips a  

year deployed into clouds, you've got another  3X in performance, another 50% or 2X in price,  

then the Hopper is only worth $0.70/hour. So the price of a GPU would continue to  

fall. That's one lens. The other lens is,  what is the utility you get out of the chip? 

If you could build infinite Rubin  or infinite of the newest chip,  

then yes, that's exactly what would happen. The price of a Hopper would fall at a spot  

or short-term contract rate as the new chips  come out and the price per performance goes up. 

But because you are so limited on semiconductors  and deployment timelines, what actually prices  

these chips is not the comparative thing I  can buy today, but rather what is the value  

I can derive out of this chip today. In that sense, let's take GPT-5.4. 

GPT-5.4 is both way cheaper to run than  GPT-4 and has fewer active parameters. 

It's much smaller, in that sense  of active parameter, because it's a  

sparser MoE versus GPT-4 being a coarser MoE. There's also been so many other advancements  

in training, RL, model architecture, and data  qualities that have made GPT-5.4 way better than  

GPT-4. And it's cheaper to serve. When you look  at an H100, it can serve more tokens per GPU of  

5.4 than if you had ran GPT-4 on it. So it's producing more tokens of a  

model that is of higher quality. What is the maximum TAM for GPT-4 tokens? 

Maybe it was a few billion dollars, maybe  it was tens of billions of dollars. Adoption  

takes time. For GPT-5.4, that number  is probably north of a hundred billion. 

But there's an adoption lag, there's  competition, and there's the constant  

improvements that everyone else is having. If improvements stopped here, the value of  

an H100 is now predicated on the value  that GPT-5.4 can get out of it instead  

of the value that GPT-4 can get out of it. These labs are in a competitive environment,  

so their margins can't go to infinity. You sort of have this dynamic that is  

quite interesting in that an H100 is worth  more today than it was three years ago. 

That's crazy. It's also interesting from  the perspective of just taking that forward. 

If we had actual AGI models developed, if  we had a genuine human on a server… These  

are such hand wave-y numbers about  how many flops the brain can do. 

But on a flop basis, an H100 is estimated to  do 1e15, which is how much some people estimate  

the human brain does in flops. Obviously, in terms of memory,  

the human brain has way more. An H100 is 80 gigabytes,  

and the brain might have petabytes. Oh, yeah, you've got petabytes? Name a petabyte  

of ones and zeros, bro. Name me a string. Well, this is actually the point. 

No, we’ve just got the best  sparse attention techniques ever. 

Genuinely though. In the amount of information  that is compressed, it might be petabytes. 

The brain is an extremely sparse MoE. But anyways, imagine a human knowledge  

worker can produce six figures a year of value. If an H100 can produce something close to that,  

if we had actual humans on a server, the  value of an H100 is such that it can repay  

itself in the course of a couple of months. So when I interviewed Dario, the point I was  

trying to make is not that I think the singularity  is two years away and therefore Dario desperately  

needs to buy more compute, although the revenue is  certainly there that he needs to buy more compute. 

The point I was trying to make is that given what  Dario seems to be saying—given his statements that  

we're two years away from a data center of  geniuses, and certainly not more than five  

years away, and a data center of geniuses should  be earning trillions upon trillions of dollars  

of revenue—it just does not make sense why  he keeps making these statements about being  

more conservative on compute or, to your point,  being less aggressive than OpenAI on compute. 

I guess that point got lost because then people  were roasting me, saying, "Oh, this podcaster  

is trying to convince this multi-hundred  billion dollar company CEO to YOLO it, bro." 

I was just trying to say that internally,  his statements are inconsistent. 

Anyway, it's good to iron it out. I think going back to the earlier  

view that if the models are so powerful, the  value of a GPU goes up over time, right now  

only OpenAI and Anthropic have that viewpoint. But as we approach further out, everyone is going  

to be able to see that value skyrocket per GPU. So in that sense,  

you should commit now to compute. Interestingly, in Anthropic fashion,  

there's a bit of a meme that they have  commitment issues and are sort of polyamorous. 

Not Dario, but this is a bit of a meme. Explains everything. By the way, there's  

this interesting economic effect called  Alchian-Allen, which is the idea that if  

you increase the fixed cost of different goods,  one of which is higher quality and one which is  

lower quality, that will make people choose  the higher quality good, on the margin. 

To give a specific example, suppose the  better-tasting apple costs two dollars and  

the shittier apple costs one dollar. Now suppose you put an import tariff on them. 

Now it's $3 versus $2 for a great  apple versus a medium apple. 

Is that because they both increased by a  dollar, or should it be a 50% increase? 

No, because they both increased by $1. The whole effect is that if there's  

a fixed cost that is applied to both. Then the price difference between them,  

the ratio, changes. Previously, the more  

expensive one was 2X more expensive. Now it's just 1.5X more expensive. 

So I wonder if applied to AI that would mean  that, if GPUs are going to get more expensive,  

there will be a fixed cost  increase in the price of compute. 

As a result, that will push people to be willing  to pay higher margins for slightly better models. 

Because the calculus is, I'm going to be  paying all this money for the compute anyway. 

I might as well just pay slightly more to  make sure it's the very best model rather  

than a model that's slightly worse. So the Hopper went from $2 to $3. 

If a Hopper can make a million tokens of Opus  and it can make two million tokens of Sonnet,  

the price differential between Opus and  Sonnet has decreased because the price of  

the GPU has increased by a dollar from $2 to $3.  Interesting. I think that makes a ton of sense. 

We just see all of the volumes  are on the best models today,  

all the revenue is on the best models today. In a compute-limited world, two things happen. 

One, companies that don't have commitment issues  and have these five-year contracts for compute  

have locked in a humongous margin advantage. They've locked in compute for five years  

at the price it transacted at  two, three, or five years ago. 

Whereas if you're three years into that  five-year contract and someone else's  

two-year or three-year contract rolled off, and  now they're trying to buy that at modern pricing,  

when it's priced to the value of models,  the price is going to be up a lot more. 

So the person who committed early  has better margins in general. 

The percentage of the market that is in long-term  contracts is much larger than the percentage of  

the market in short-term contracts that can be  this flex capacity you add at the last second. 

At the same time, where does the margin go? Because models get more valuable,  

how much can the cloud players flex their pricing? 

If you look at CoreWeave, their average  term duration is over three years right now. 

For ninety-eight percent plus of  their compute, it's over three years. 

They end up with this conundrum  where they can't actually flex price. 

But every year they're adding incrementally  way more capacity than they had previously. 

This year alone, Meta's adding as much capacity as  they had in their entire fleet of compute and data  

centers for all purposes for serving WhatsApp,  Instagram, and Facebook in 2022, and doing AI. 

They're adding that alone this year. In the same sense, you talk about Meta doing that,  

CoreWeave, Google, and Amazon, all these companies  are adding insane amounts of compute year on year. 

That new compute gets transacted at the new price. In a sense, yes, you've locked in, as long as  

we're in a takeoff. "Oh, OpenAI went from six  hundred megawatts to two gigawatts last year,  

and from two gigawatts to six plus this  year, and six to twelve next year." 

The incremental added compute is where all the  cost is, not the prior long-term contracts. 

Then who holds the cards is the  infra providers for charging margin. 

Now the cloud players, the neoclouds, or  the hyperscalers can charge the margin. 

They can to some extent, but then as you go  upstream to who has access to all the memory and  

logic capacity, it's Nvidia for the most part. They've signed a lot of long-term contracts. 

They've got ninety billion dollars of long-term  contracts today, and they're negotiating  

three-year deals today with the memory vendors. You've got Amazon and Google through Broadcom,  

Amazon directly, and AMD. These companies hold all the  

cards because they've secured the capacity. TSMC is not raising prices, but memory vendors  

are, to some extent, raising a lot of price. They're going to double or triple price again, but  

then they're also signing these long-term deals. Who is able to accrue all the margin dollars is  

potentially the cloud, potentially the  chip vendors, and the memory vendors,  

until TSMC or ASML break out and say,  "No, we're going to charge a lot more." 

But at the same time, do the model  vendors get to charge crazy margins? 

At least this year, we're going to see  margins for the model vendors go up a lot. 

Because they're so capacity constrained,  they have to destroy demand. 

There's no way Anthropic can continue at  the current pace without destroying demand. 

Let's get into logic and memory. How specifically has Nvidia been  

able to lock up so much of both? I think according to your numbers,  

by '27, Nvidia is going to have +70% of  N3 wafer capacity, or around that area. 

I forget what the numbers were for memory  at SK Hynix and Samsung and so forth. 

Think about how the neocloud business  works and how Nvidia works with that,  

or how the RL environment business  works and how Anthropic works with that. 

In both those cases, Nvidia is purposely trying to  fracture the complementary industry to make sure  

that they have as much leverage as possible. They're giving allocation to random neoclouds  

to make sure that there's not one  person that has all the compute. 

Similarly, Anthropic or OpenAI, when they're  working with the data providers, they say, "No,  

we're going to just seed a huge industry  of these things so that we're not locked  

into any one supplier for data environments." And I wonder why on the 3 nm process—that's  

going to be Trainium 3, that's going to be  TPU v7, other accelerators potentially—why  

is TSMC just giving it all up to Nvidia  rather than trying to fracture the market? 

There are a couple points here. On 3 nm, if we go back to last year,  

the vast majority of 3 nm was Apple. Apple is being moved to 2 nm. 

Memory prices are going up, so  Apple's volumes may go down. 

As memory prices go up, either  they cut margin or they move on. 

There's some time lag because they have  long-term contracts, but Apple likely  

reduces demand or moves to 2 nm faster, where  2 nm is only capable of mobile chips today. 

In the future, AI chips will move there.  So Apple has that. Apple is also talking to  

third-party vendors because they're  getting squeezed out of TSMC a little bit. 

TSMC's margins on high-performance computing—HPC,  AI chips, et cetera—are higher than they are  

for mobile, because they have a bigger  advantage in HPC than they do in mobile. 

When you look at TSMC’s running calculus  here, they're actually providing really good  

allocations to companies that are doing CPUs. When you think about Amazon having Trainium and  

Graviton, both of those are on 3 nm, Graviton  being their CPU, Trainium being their AI chip. 

TSMC is much more excited to give  allocation to Graviton than they  

are to Trainium because they view the CPU  business as more stable, long-term growth. 

As a company that is conservative and doesn't  want to ride cycles of growth too hard,  

you actually want to allocate to the market that  is more stable with a lower growth rate first  

before you allocate all the incremental capacity  to the fast growth rate market. That is the case  

generally. Same for AMD. The allocations they  get on their CPUs, TSMC is much more excited  

about those than they are for GPUs. Likewise  for Amazon. Nvidia is a bit unique because yes,  

they have CPUs, they make switches, they make  networking, NVLink, InfiniBand, Ethernet, NICs. 

By and large, most of these things will  be on 3 nm by the end of this year with  

the Rubin launch and all the chips in that  family, the GPU being the most important one. 

Yet Nvidia is getting the majority of supply. Part of this is because you look at the market  

and TSMC and others forecast market demand in  many ways, but it's also the market signal. 

The market signaled, "Hey, we need this much  capacity next year. We need this much. We'll  

sign non-cancelable, non-returnable. We  may even pay deposits." Nvidia just did  

it way earlier than Google or Amazon. In some cases, Google and Amazon had  

stumbling blocks. One of the chips  

got delayed slightly by a couple quarters. Trainium and all these sorts of things happened. 

In that case, there was a huge sort  of, "Well, these guys are delaying,  

but Nvidia is wanting more, more, more, more. And we are checking with the rest of the supply  

chain, is there enough capacity?" They're going to all the PCB  

vendors and saying, "Is there enough PCB?" Victory Giant is one of the largest suppliers  

of PCBs to Nvidia, and they're a Chinese company. All the PCBs come from China, or many of them. 

They're like, "Do you have enough PCB capacity?  Great. Hey memory vendors, who has all the memory  

capacity? Okay, Nvidia does. Great." When you  look at who is AGI-pilled enough to buy compute  

on long timelines at levels that seem ridiculous  to people who aren't AGI-pilled—but nonetheless,  

they're willing to pay a pretty good margin and  sign it now because they view in the future that  

ratio is screwed up—the same thing happens  with the supply chain for semiconductors. 

I don't think Nvidia is quite AGI-pilled. Jensen doesn't believe software is going  

to be fully automated and all these things. Accelerated computing, not AI chips, right? 

It's AI chips. But that's what he calls it, right? 

Yeah. I think it's a broader term, AI is within  that, but also physics modeling and simulations. 

But it's like he's not  embracing the main use case. 

I think he's embracing it, but I just don't  think he's AGI-pilled like Dario or Sam. 

But he's still way, way more AGI-pilled than  Google was in Q3 of last year, or Amazon was  

in Q3 of last year, and he saw way more demand.  The reason is pretty simple. You can see all the  

data center construction. He's like, "Okay,  

I want to have this market share." We have all the data centers tracked,  

and there's a lot of data centers  that could be one or the other. 

To some extent, Google and Amazon, Google  especially, even though their TPU is just  

better for them to deploy, they have to  deploy a crap load of GPUs because they  

don't have enough TPUs to fill up their  data centers. They can't get them fabbed. 

I have a question about that. Google sold a million, was it  

the v7s? Yes. 

—the Ironwoods to Anthropic, and you're saying the  big bottleneck right now, this year or next year,  

I guess going forward forever now,  is going to be the logic and memory,  

the stuff it takes to build these chips. Google has DeepMind, the third prominent AI lab. 

If this is the big bottleneck, why would they  sell it rather than just giving it to DeepMind? 

This is again a problem of… DeepMind people  were like, "This is insane. Why did we do  

this?" But Google Cloud people and Google  executives saw a different thought process. 

You and I know the compute team at Anthropic. Both of the main people came from Google. 

They saw this dislocation, they negotiated  a deal, and they were able to get access  

to this compute before Google realized. The chain of events, at least from our  

data that we found, was in early Q3, over  the course of six weeks, we saw capacity  

on TPUs go up by a significant amount. It went up multiple times in those six  

weeks. There were multiple requests. Google  even had to go to TSMC and explain to them  

why they needed this increase in  capacity because it was so sudden. 

A lot of that capacity increase  was for selling to Anthropic. 

Because Anthropic saw it before Google. And then Google had Nano Banana and Gemini  

3 which caused their user metrics to skyrocket. Then leadership at Google was like, "Oh." 

Then they started making the statement that  we have to double compute every six months,  

or whatever the exact number was. They really woke up a lot more, and then  

they went to TSMC and said, "We want more. We want  more." TSMC replied, "Sorry guys, we're sold out. 

We can maybe get 5-10% more for 2026,  but really we're going to work on 2027." 

There was this information asymmetry among  the labs, in my mind. I don't know exactly.  

It's the narrative I've spun myself from  seeing all the data in the supply chain on  

wafer orders and what's going on with the data  centers that Anthropic and Fluidstack signed. 

It's pretty clear to me that Google screwed up. You can see this from Google's Gemini ARR. 

They had next to nothing in Q1 to Q3—in Q3  a little bit once they started inflecting. 

But in Q4 they reached $5 billion  in revenue on an ARR basis. 

It's clear Google didn't see  revenue skyrocket initially. 

In a sense, Anthropic had a little bit of  commitment issues before their ARR exploded,  

even though they had far more information  asymmetry and saw what was coming down the pipe. 

Google is going to be more conservative  than Anthropic and Google had even less ARR. 

So they were just not willing to do it,  and then they realized they should do it. 

Since then, Google has gotten absurdly  AGI-pilled in terms of what they're doing.  

They bought an energy company. They're  putting deposits down for turbines. 

They're buying a ridiculous  percentage of powered land. 

They're going to utilities and  negotiating long-term agreements. 

They're doing this on the data center  and power side very aggressively. 

I think Google woke up towards the end  of last year, but it took them some time. 

How many gigawatts do you think Google  will have by the end of next year? 

Buy my data. You charge for that kind of information. 

Yes, yes. I feel like every year the  bottleneck for what is preventing us  

from scaling AI compute keeps changing. A couple years ago it was CoWoS. Last  

year it was power. You'll tell me  what the bottleneck is this year. 

But I want to understand five years  out, what will be the thing that is  

constraining us from deploying the singularity? The biggest bottleneck is compute. For that,  

the longest lead time supply chains  are not power or data centers. 

They're actually the semiconductor  supply chains themselves. 

It switches back from power and data  centers as a major bottleneck to chips. 

In the chip supply chain, there's  a number of different bottlenecks. 

There's memory, logic wafers from  TSMC, and the fabs themselves. 

Construction of the fabs takes two to three years,  versus a data center which takes less than a year. 

We've seen Amazon build data  centers in as fast as eight months. 

There's a big difference in lead  times because of the complexity  

of building the fab that actually makes the chips. 

The tools also have really long lead times. The bottlenecks, as we've scaled,  

have shifted based on what the supply  chain is currently not able to do. 

It was CoWoS, power, and data centers, but  those were all shorter lead time items. 

CoWoS is a much simpler process  of packaging chips together. 

Power and data centers are ultimately way simpler  than the actual manufacturing of the chips. 

There's been some sliding of capacity  across mobile or PC to data center chips,  

which has been somewhat fungible. Whereas CoWoS, power, and data  

centers have had to start anew as supply chains. But now there's no more capacity for the mobile  

and PC industries—which used to be the majority  of the semiconductor industry—to shift over to AI. 

Nvidia is now the largest customer at TSMC  and SK Hynix, the largest memory manufacturer. 

It's sort of impossible for the  sliding of resources away from  

the common person's PCs and smartphones  to shift any more towards the AI chips. 

So now the question is how do  we scale AI chip production? 

That's the biggest bottleneck as we go to 2030. It would be very interesting if there's an  

absolute gigawatt ceiling that you can project  out to 2030 based just on "We can't produce more  

than this many EUV machines." To scale compute further,  

there are different bottlenecks this year and  next year, but ultimately by 2028 or 2029,  

the bottleneck falls to the lowest rung  on the supply chain, which is ASML. 

ASML makes the world's most  complicated machine: an EUV tool. 

The selling price for those is $300-400 million. Currently, they can make about 70. 

Next year, they'll get to 80. Even under very aggressive supply  

chain expansion, they only get to a little  bit over 100 by the end of the decade. What  

does that mean? They can make a hundred of these  tools by the end of the decade, and 70 right now. 

How does that actually translate to AI compute? We see all these numbers from Sam Altman and  

many others across the supply chain:  gigawatts, gigawatts, gigawatts. 

How many gigawatts are we adding? We see Elon saying a hundred gigawatts in space. 

A year. A year. The problem with any of  

these numbers, or the challenge to these numbers,  is actually not the power or the data center. 

We can dive into that, but  it's manufacturing the chips. 

Take a gigawatt of Nvidia's Rubin chips. Rubin is announced at GTC,  

I believe the week this podcast goes live. To make a gigawatt worth of data center  

capacity of Nvidia's latest chip that they're  releasing towards the end of this year,  

you need a few different wafer technologies. You need about 55,000 wafers of 3 nm. 

You need about 6,000 wafers of 5 nm, and then  you need about 170,000 wafers of DRAM memory. 

Across these three different buckets,  each requires different amounts of EUV. 

When you manufacture a wafer, there are thousands  and thousands of process steps where you're  

depositing material and removing them. But the key critical step—which at least  

in advanced logic is 30% of the  cost of the chip—is something that  

doesn't actually put anything on the wafer. You take the wafer, you deposit photoresist,  

which is a chemical that chemically  changes when you expose it to light. 

Then you stick it into the EUV tool, which  shines light at it in a certain way. It  

patterns it. There's what's called a mask,  which is effectively a stencil for the design. 

When you look at a leading-edge 3 nm wafer, it has  70 or so masks, 70 or so layers of lithography,  

but 20 of them are the most advanced EUV. If you need 55,000 wafers for a gigawatt, and you  

do 20 EUV passes per wafer, you can do the math. That's 1.1 million passes of EUV for a single  

gigawatt. It's pretty simple. Once you add the  rest of the stuff, it ends up being 2 million,  

across 5 nm and all the memory. You're at roughly 2 million EUV  

passes for a single gigawatt. These tools  are very complicated. When you think about  

what it's doing across a wafer, it's taking  the wafer and scanning and stepping across. 

It does this dozens of times  across the whole wafer. 

When you're talking about how  many EUV passes, that’s the  

entire wafer being exposed at a certain rate. An EUV tool can do roughly 75 wafers per hour,  

and the tool is up roughly 90% of the time. In the end, you need about three and a half  

EUV tools to do the 2 million EUV  wafer passes for the gigawatt. 

So three and a half EUV  tools satisfies a gigawatt. 

It's funny to think about the numbers. What does  a gigawatt cost? It costs roughly $50 billion.  

Whereas what do three and a half EUV tools cost?  That's $1.2 billion. It's actually quite a lower  

number, which is interesting to think about. Fifty gigawatts of economic CapEx in the data  

center, and what gets built on top of  that in terms of tokens is even larger. 

It might be $100 billion worth of  AI value into the supply chain,  

three years, TSMC has done $100 billion of CapEx.  So it's $30/$30/$40 billion. A small fraction of  

that is being used by Nvidia for the 3 nm, or  previously 4 nm, that it's using for its chips. 

What were its earnings last  quarter? It was $40 billion.  

So $40 billion times four is $160 billion. Nvidia alone is turning some small fraction  

of $100 billion in CapEx, which is going to  be depreciated over many years and not just  

this one year, into $160 billion in a single year. That gets even more intense when you go down the  

supply chain to ASML, which is taking a billion  dollars' worth of machines to produce a gigawatt. 

Of course, those machines last for more  than a year so it’s doing more than that. 

Now I want to understand, how many  such machines will there be by 2030,  

if you include not just the ones that are sold  that year, but have been compiling over the  

previous years? What does that imply? Sam Altman  says he wants to do a gigawatt a week in 2030. 

When you add up those numbers,  is it compatible with that? 

That's completely compatible,  if you think about it. 

TSMC and the entire ecosystem have  something like 250 to 300 EUV tools already. 

Then you stack on 70 this year, 80  next year, growing to 100 by 2030. 

You're at 700 EUV tools by the end of the decade.  700 EUV tools, at three and a half tools per  

gigawatt—assuming it's all allocated to AI, which  it's not—gets you to 200 gigawatts worth of AI  

chips for the data centers to deploy. Sam wants 52 gigawatts a year. 

He's only taking 25% share then. Obviously, there's some share given to mobile and  

PC, assuming we're even allowed to have consumer  goods still and we don't get priced out of them. 

But roughly, he's saying 25% market  share of the total chips fabbed. 

That's very reasonable given that this  year alone, I think he's going to have  

access to 25% of the Blackwell GPUs  that are deployed. It's not that crazy. 

When did ASML start shipping  EUV tools, when 7 nm started? 

I don't know when that was exactly. You're saying in 2030, they're going to be using  

machines that initially were shipped in 2020. So for ten years, you're using the same most  

important machine in this  most technologically advanced  

industry in the world? I find that surprising. ASML's been shipping EUV tools now for roughly a  

decade, but it only entered mass volume production  around 2020. The tool's not the same. Back then,  

the tools were even lower throughput. There are various specifications around  

them called overlay. I was mentioning you're  

stacking layers on top of each other. You'll do some EUV, you'll do a bunch  

of different process steps—depositing stuff,  etching stuff, cleaning the wafer—dozens of  

those steps before you do another EUV layer. There's a spec called overlay, which is:  

you did all this work, you drew these lines  on the wafer, now I want to draw these dots. 

Let's say I want to draw these dots to  connect these lines of metal to holes,  

and then the next layer up is another set of lines  going perpendicular, so now you're connecting  

wires going perpendicular to each other. You have to be able to land them on top of  

each other. It's called overlay. Overlay is  a spec that's been improved rapidly by ASML. 

Wafer throughput has been  improved rapidly by ASML. 

The price of the tool has gone up, but not  as much as the capabilities of the tool. 

Initially, the EUV tools were $150 million. Over time, they're now $400 million  

as I look out to 2028. But the capabilities of the  

tools have more than doubled as well, especially  on throughput and overlay accuracy, which is  

the ability to accurately align the subsequent  passes on top of each other even though you do  

tons of steps between. ASML is improving super  rapidly. It's also noteworthy to say that ASML  

is maybe one of the most generous companies  in the world. They have this linchpin thing.  

No one has anything competitive. Maybe China will  have some EUV by the end of the decade, but no one  

else has anything even close to EUV, and yet they  haven't taken price and margins up like crazy. 

You go ask some other folks that we  talk to all the time, like Leopold,  

and they're like, "Let's have the price go  up." Because they can. The margin is there.  

You can take the margin. Nvidia takes the  margin. Memory players are taking the margin. 

But ASML has never raised the price more than  they've increased the capability of the tool. 

In a sense, they've always provided  net benefit to their customers. 

It's not that the tool is stagnant,  it's just that these tools are old. 

Yes, you can upgrade them some,  and the new tools are coming. 

For simplicity's sake, we're  ignoring the advances in overlay  

or throughput per tool for this podcast. You say we're producing 60 of these machines  

this year and then 70, 80 over subsequent years. 

What would happen if ASML just decided  to double its CapEx or triple its CapEx? 

What is preventing them from  producing more than 100 in 2030? 

Why are you so confident that  even five years out, you can be  

relatively sure what their production will be? I think there are a couple factors here. 

ASML has not decided to just go YOLO,  let's expand capacity as fast as possible. 

In general, the semiconductor  supply chain has not. 

It's lived through the booms and busts,  and we can talk a bit more about it. 

Basically some players have recently woken  up, but in general no one really sees demand  

for 200 gigawatts a year of AI chips, or  trillions of dollars of spend a year in  

the semiconductor supply chain. They're  not AI-pilled. They're not AGI-pilled. 

We're going to get to a  trillion dollars this year. 

Yeah, I feel you, but I'm saying no one  really understands this in the supply chain. 

Constantly, we're told our numbers are  way too high, and then when they're right,  

they're like, "Oh, yeah, but your next  year's numbers are still too high." 

ASML's tool has four major components. It has the source,  

which is made by Cymer in San Diego. It has the reticle stage, which is made  

in Wilmington, Connecticut. It has the wafer  stage. It has the optics, the lenses and such. 

Those last two are made in Europe. When you look at each of these four,  

they're tremendously complex supply chains that,  (A) they have not tried to expand massively,  

and (B) when they try to expand  them, the time lag is quite long. 

Again, this is the most complicated machine  that humans make, period, at any sort of volume. 

Let's talk about the source specifically. What  does the source do? It drops these tin droplets.  

It hits it three subsequent  times with a laser perfectly. 

The first one hits this tin  droplet, it expands out. 

It hits it again, so it expands  out to this perfect shape,  

and then it blasts it at super high power. The tin droplets get excited enough that they  

release EUV light, 13.5 nanometer, and then  it's in this thing that is collecting all  

the light and directing it into the lens stack. Then you have the lens stack, which is Carl Zeiss,  

as you mentioned, and some other folks, but  Zeiss being the most important part of it. 

They also have not tried to expand  production capacity because they don't see... 

They're like, "We're growing a lot because of AI. We're growing from 60 to 100." It's like, "No, no,  

no. We need to go to a couple hundred, but it's  fine. Whatever." Each of these tools has, I think,  

18 of these lenses, effectively. They are multilayer mirrors,  

which are perfect layers of molybdenum  and ruthenium, if I recall correctly,  

stacked on top of each other in many layers,  and then the light bounces off of it perfectly. 

When we think about a lens, it's in  a shape, and it focuses the light. 

This is like a mirror that's also  a lens, so it's pretty complicated. 

Any defect in these super thinly  deposited stacks will mess it up. 

Any curvature issues will mess it up. There are a lot of challenges  

with scaling the production. It's quite artisanal in this sense  

because you're not making tens of thousands  of these a year, you're making hundreds,  

you're making thousands. 60 tools a year, 18 of  these per tool, you’re still in the hundreds,  

of tools, or you're at the thousand number  roughly for these lenses and projection optics. 

Then you step forward to the reticle stage,  which is also something really crazy. 

This thing moves at, I want to say, nine Gs. It will shift nine Gs because as you step  

across a wafer, the tool will go... The  wafer stage is complementary. It's the  

wafer part. You line these two things up. You're taking all the light through the  

lenses that's focused, and here's  the reticle, here's the wafer. 

The reticle's moving one direction, the  wafer's moving the other direction as it  

scans a 26x33 millimeter section  of the wafer, and then it stops. 

It shifts over to another part  of the wafer and does it again. 

It does that in just seconds. Each of them is moving  

at nine Gs in opposite directions. Each of these things is a wonder and marvel  

of chemistry, fabrication, mechanical engineering,  and optical engineering, because you have to align  

all these things and make sure they're perfect. All of these things have crazy amounts  

of metrology because you have  to perfectly test everything. 

If anything is messed up, the yield goes to  zero, because this is such a finely tuned system. 

By the way, it's so large that you're building  it in the factory in Eindhoven, Netherlands,  

and they're deconstructing it and shipping it on  many planes to the customer site, and then you're  

reassembling it there and testing it again. That process takes many, many months. 

There are so many steps in the supply  chain, whether it's Zeiss making their  

lenses and projection optics or Cymer, which is  an ASML-owned company, making the EUV source. 

Each of these has its own complex supply chain. ASML has commented that their supply chain has  

over ten thousand people in it. Like individual suppliers? 

Yes. It might not be directly. It might  be through Zeiss having so many suppliers  

and XYZ company having so many suppliers. If you just think about it, you're talking  

about two physically moving objects that are  the size of a wafer, and it has to be accurate  

to the level of single-digit nanometers or even  smaller because the entire system, the overlay,  

the layer-to-layer overlay variation,  has to be on the order of 3 nanometers. 

If the overlay is 3 nms, that means each  individual part, the accuracy of its  

physical movement has to be even less than that. It has to be sub-one nanometer in most cases,  

because the error of these things stack up. There's no way to just snap your fingers and  

increase production. Things as simple as power.  The US going from zero percent power growth to  

two percent power growth, even though China's  already at thirty, was so hard for America to do. 

And that's a really simple supply chain with  very few people in it who make difficult things. 

There are probably 100,000  electricians and people who work in  

the electricity supply chain, or more, in the US? When you look at ASML, they employ so few people. 

Carl Zeiss probably employs less than a  thousand people working on this, and all of  

those people are super, super specialized. You can't just train random people up  

for this in the snap of a finger. You can't just get your entire supply  

chain to get galvanized. Nvidia's had to do a lot  

to get the entire supply chain to even deliver  the capacity they're going to make this year. 

When you go talk to Anthropic, they're  like, "We're short of TPUs, we're short  

of training, and we're short of GPUs." When you go talk to OpenAI, they're like,  

"We're short of these things." OpenAI and Anthropic know they need X. 

Nvidia is not quite as AGI-pilled.  They're building X - 1. You go down the  

supply chain, everyone's doing X - 1. In some cases, they're doing X ÷ 2,  

because they're not AGI-pilled. You end up with this time lag  

for the whip to react. The AI-pilledness and the  

desire to increase production takes so long. Once they finally understand that they need  

to increase production rapidly…  They think they understand. 

They think AI means we have to go from 60  to 100, in addition to the tools getting  

better and faster, the source getting  higher power from 500 watts to 1,000,  

and all these other aspects of the supply chain  advancing technically and increasing production. 

They think they're actually  increasing production a lot. 

But if you flow through the  numbers… What does Elon want? 

He wants 100 gigawatts a year  in space by 2028 or 2029. 

Sam Altman wants 52 gigawatts a  year by the end of the decade. 

Anthropic probably needs the  same, and Google needs that. 

You go across the supply chain, and it's  like, wait, no, the supply chain can't  

possibly build enough capacity for everyone  to get what they want on the side of compute. 

I feel like in the data center  supply chain for the last few years,  

people have been making arguments like, "We are  bottlenecked by this specific thing, therefore  

AI compute can't scale more than X." But as you've written about, if the  

grid is a bottleneck, then we just do behind the  meter on the site, we do gas turbines, et cetera. 

If that doesn't work, there are all these  other alternatives that people fall back on. 

I want to ask whether we can imagine a similar  thing happening in the semiconductor supply chain. 

If EUV becomes a bottleneck, what if we  just went back to 7 nm and did what China  

is doing currently, producing 7 nm chips  with multi-patterning with DUV machines? 

If you look at a 7 nm chip like the  A100, there's been a lot of progress  

obviously from the A100 to the B100 or B200. How much of that progress is just numerics? 

If you just hold FP16 constant from A100 to B100. The B100 is a little over one petaflop, and the  

A100 is like 300 teraflops. Yeah, 312. 

Holding numerics constant, you have  a 3x improvement from A100 to B100. 

Some of that is the process improvement, some of  that is just the accelerator design improving,  

which we could replicate again in the future. It seems there's actually a very small effect  

from the process improving from 7nm to 4 nm. I don't know the numbers offhand but let's  

say there's 150k wafers per month of 3 nm  and eventually similar amounts for 2 nm. 

But then there's a similar amount for 7 nm. If you have all those old wafers and there's  

maybe a 50% haircut because the bits per  wafer area are 50% less or something,  

it doesn't seem that bad to just bring on 7  nm wafers if that gives you another fifty or  

hundred gigawatts. Tell me why that's naive. We potentially do go crazy enough that this  

happens because we just need incremental  compute, and the compute is worth the  

higher cost and power of these chips. But it's also unlikely to a large extent  

because some of these are not fair comparisons. For example, from A100, which is 312 teraflops,  

to Blackwell, which is 1,000 or 2,000 FP16,  and then Rubin is 5,000 or so FP16… It's not  

a fair comparison because these chips  have vastly different design targets. 

With A100, Nvidia optimized  for FP16 and BF16 numerics. 

When you look at Hopper, they didn't care  as much about that; they cared about FP8. 

When you look at Rubin, they don’t  care about FP16 and BF16 so much,  

they care mostly about FP4 and FP6. Numerics are what they've designed their chip for. 

Let's say we make a new chip design on 7 nm,  optimized for the numerics of the modern day. 

The performance difference is  still going to be much larger  

than the FLOPS difference you mentioned. Often it's easy to boil things down to FLOPS  

per watt or FLOPS per dollar,  but that's not a fair comparison. 

Let's look at Kimi K2.5 and DeepSeek. When you look at those two models and  

their performance on Hopper versus  Blackwell on very optimized software,  

you get vastly different performance. Most of this is not attributed to  

FLOPS or numerics, because those  models are actually eight-bit. 

So it's not like Blackwells and Hopper are both  optimized for eight-bit, and Blackwell is not  

really taking advantage of its four-bit there. The performance gulf is actually much larger. 

Sure it's one thing to shrink process  technology and make the transistor smaller  

so each chip has X number of FLOPS,  but you forget the big gating factor. 

These models don't run on a single chip. They run on hundreds of chips at a time. 

If you look at DeepSeek's production  deployment, which is well over a  

year old now, they were running on 160 GPUs. That's what they serve production traffic on. 

They split the model across 160 GPUs. Every time you cross the barrier from one  

chip to another, there is an efficiency loss. You have to transmit over high-speed  

electrical SerDes, which brings  a latency cost and a power cost. 

There are all these dynamics that hurt. As you shrink and shrink the process node, you've  

increased the amount of compute in a single chip. Now in-chip movement of data is at least tens  

of terabytes a second, if not  hundreds of terabytes a second. 

Whereas between chips, you're on  the order of a terabyte a second. 

Then you have this movement of data between chips  that are super close to each other physically. 

You can only put so many chips  close to each other physically,  

so you have to put chips in different racks. The movement of data between racks is on the order  

of hundreds of gigabits a second, 400 gig or 800  gig a second, so roughly 100 gigabytes a second. 

So you have this huge ladder: on-chip  communication is super fast, within the  

rack is an order of magnitude slower, and outside  the rack is an order of magnitude lower than that. 

As you break the bounds of chips,  you end up with a performance loss. 

The reason I explain this is because  when you look at Hopper versus Blackwell,  

even if both are using a rack's worth of  chips, Hopper is significantly slower. 

The amount of performance you have leveraged to  the task within each domain—tens of terabytes a  

second of communication between these processing  elements versus terabytes a second between these  

processing elements—is much, much higher and  therefore the performance is much higher. 

When you look at inference at 100 tokens  a second for DeepSeek and Kimi K2.5,  

the performance difference between Hopper  and Blackwell is on the order of 20x. 

It's not 2x or 3x like the FLOPS  performance difference indicates,  

even though those are on the same process node. There are just differences in networking  

technologies and what they've worked on. You can translate some of these back,  

but when you look at what they're doing  on 3 nm with Rubin, some of those things  

are simply not possible to do all the way back  on A100, even if you make a new chip for 7 nm. 

There are certain architectural improvements  you can port and certain ones you cannot. 

The performance difference is not just  going to be the difference in FLOPS. 

It's in some senses cumulative between  the difference in FLOPS per chip,  

networking speed between chips, how many FLOPS  are on a chip versus a system, and memory  

bandwidth on a single chip versus an entire  system. All of these things compound. 

Can I ask you a very naive question? The B200 now has two dies on a single chip,  

so you can get that bandwidth without  having to go through NVLink or InfiniBand. 

Next year, Rubin Ultra will  have four dies on one chip. 

What is preventing us from just doing  that with an older… How many dies could  

you have on a single chip and still  get these tens of terabytes a second? 

Even within Blackwell, there are differences in  

performance when you're communicating  on the chip versus across the chips. 

Those bounds are obviously much smaller than  when you're going out of the entire chip. 

When you scale the number of chips  up, there is some performance loss. 

It's not perfect, but it is way  better than different entire packages. 

How large can advanced packaging scale? The way Nvidia is doing it is CoWoS. 

Google, Broadcom, MediaTek, and  Amazon's Trainium are all doing CoWoS. 

But actually you can go look back at what Tesla  did with Dojo, which they cancelled and restarted. 

Dojo was a chip that was  the size of an entire wafer. 

They had 25 chips on it. There were some  tradeoffs. They couldn't put HBM on it. 

But the positive side was  that they had 25 chips on it. 

To date, it is still probably the best chip  for running convolutional neural networks. 

It's just not great at transformers because the  shape of the chip, the memory, the arithmetic,  

and all these various specifications are just  not well-suited for transformers. They're  

well-suited for CNNs. Dojo chips were optimized  around that, and they made a bigger package. 

But as you make packages bigger and bigger,  you have other constraints: networking speed,  

memory bandwidth, and cooling capabilities. All of these things start to rear their heads.  

It's not simple. But yes, you will see a trend  line of more chips on the package, and yes,  

you're going to be able to do that on 7 nm. In fact, that's what Huawei did  

with their Ascend 910C or D. They initially put one, and then they did two. 

They're focusing on scaling the  packaging up because that is an  

area where they can advance faster than  process technology where they can't shrink. 

But at the end of the day, that’s something  you can do on the leading-edge chips too. 

Anything you do on 7 nm, you can also  probably do on 3 nm in terms of packaging. 

If we end up in this world in 2030 where the  West has the most advanced process technology  

but has not ramped it up as much, whereas  China… I don't know if you think by 2030  

they would have EUV and 2 nm or whatever. But they are semiconductor-pilled and they  

are producing in mass quantity. Basically, I'm wondering what  

the year is where there's a crossover, where our  advantage in process technology has faded enough,  

and their advantage in scale has increased enough. And also, if their advantage in having one country  

with the entire supply chain indigenized—rather  than having random suppliers in Germany  

and the Netherlands—would mean that China would  be ahead in its ability to produce mass flops. 

To date, China still does not have an entirely  indigenized semiconductor supply chain. 

But would they in 2030? By 2030, it's possible that they do. 

But to date, all of China's 7 nm and  14 nm capacity uses ASML DUV tools. 

The amount that they can  import from ASML is large. 

But the vast majority of ASML's revenue,  especially on EUV all of it, is outside of China. 

The scale advantage is still in  the favor of the West plus Taiwan,  

Japan, and Korea, et cetera. But they're trying to make  

their own DUV and EUV tools, right? They're trying to do all these things. 

The question is how fast can they advance  and scale up production as well as quality. 

To date, we haven't seen that. Now I'm quite bullish that they're  

going to be able to do these things  over the next five to ten years. 

They will really scale up production  and kick it into high gear. 

They have more engineers working on it and  more desire to throw capital at the problem. 

So by 2030, will they have fully indigenized DUV? I think for sure. DUV, yes. 

And fully indigenized EUV by 2030? I think they'll have working tools. 

I don't think that they'll be  able to manufacture a bunch yet. 

There's having it work, and  then there's production hell. 

ASML had EUV working in the  early 2010s at some capacity. 

The tools were not accurate enough. They were not scaled for high-volume  

manufacturing or reliable enough. They had to ramp production,  

and that all took time. Production hell takes  time. That's why it took another five to seven  

years to get EUV into mass production at  a fab rather than just working in the lab. 

How many DUV tools do you think  they'll be able to manufacture in 2030? 

ASML? No, China. 

That's a great question. It's a bit of a  challenge to look into this supply chain  

especially. We try really hard. In some instances,  they're buying stuff from Japanese vendors. 

If they want a fully indigenized supply chain,  they need to not buy these lenses, projection  

optics, or stages from Japanese vendors. They need to build it internally. 

It's really tough to say where  they'll be able to get to. 

I honestly think it's a shot in the dark. But it's probably not unlikely that they'll  

be able to do on the order of 100 DUV tools  a year, whereas ASML is currently doing  

hundreds of DUV tools a year. No company has a process node  

where they make a million wafers a month. Elon says he wants to do it and China is  

obviously going to do it. TSMC is trying to do that. 

The memory makers may get to a million wafers  a month as well, but not in a single fab. 

It's mind-boggling to think of  that scale, and challenging to  

see the supply chain galvanized for that. I don't want to doubt China's capability to scale. 

I guess this is an interesting question. I think at some point SemiAnalysis  

will do the deep dive on this. By when would indigenized Chinese production  

be bigger than the rest of the West combined. And put in the input of your model of when they'll  

have DUV machines and EUV machines at scale? Because there's this question around if you  

have long timelines on AI—by long meaning  2035, which is not that long in the grand  

scheme of things—should you expect a world  where China is dominating in semiconductors? 

It doesn't get asked enough  because if you're in San Francisco,  

we're thinking on timescales of weeks. If you're outside of San Francisco,  

you're not thinking about AGI at all. What if we  have AGI? What if you have this transformational  

thing that is commanding tens or hundreds  of trillions of dollars of economic growth  

and token output, but it happens in 2035? What does that imply for the West versus China? 

SemiAnalysis has got to write  the definitive model on this. 

It's really challenging when you  move timescales out that far. 

What we tend to focus on is tracking every  data center, every fab, and all the tools. 

We track where they're going, but the time  lags for these things are relatively short. 

We can only make reasonably accurate estimates  for data center capacity based on land purchasing,  

permits, and turbine purchasing. We know where all these things  

are going, that's the data we sell. As you go out to 2035, things are just  

so radically different. Your error bars get so  

large it's hard to make an estimate. But at the end of the day, if takeoff  

or timelines are slow enough, I don't see why  China wouldn't be able to catch up drastically. 

In some sense, we've got this valley where, three  to six months ago, or maybe even now, Chinese  

models are as competitive as they've ever been. I think Opus 4.6 and GPT 5.4 have really pulled  

away and made the gap a little bit bigger, but  I'm sure some new Chinese models will come out. 

As we move from selling tokens where they  provide the entire reasoning chain, to  

selling automated white-collar work—an automated  software engineer, you send them the request,  

they give you the result back, and there's a bunch  of thinking on the back end that they don't show  

you—the ability to distill out of American  models into Chinese models will be harder. 

Second, look at the scale of  the compute the labs have. 

OpenAI exited the year with  roughly two gigawatts last year. 

Anthropic will get to  two-plus gigawatts this year. 

By the end of next year, they'll  both be at ten gigawatts of capacity. 

China is not scaling their AI  lab compute nearly as fast. 

At some point, when you can't distill the  learnings from these labs into the Chinese  

models, plus with this compute race that OpenAI,  Anthropic, Google, and Meta are all racing on,  

they end up getting to a point where the model  performance should start to diverge more. 

Then look at all this CapEx  being spent on data centers. 

Amazon is spending $200  billion, Google $180 billion. 

All these companies are spending  hundreds of billions of dollars on CapEx. 

There's nearly a trillion dollars  of CapEx being invested in data  

centers in America this year, roughly. What's the return on invested capital here? 

You and I would think the return on invested  capital for data center CapEx is very high. 

If we look at Anthropic's revenues,  in January they added $4 billion. 

In February, which was a shorter  month, they added $6 billion. 

We'll see what they can do in March and April,  

given that compute constraints are  what's bottlenecking their growth. 

The reliability of Claude is quite low  because they're so compute constrained. 

But if this continues, then the ROIC  on these data centers is super high. 

At some point, the US economy starts growing  faster and faster over this year and next year  

because of all this CapEx, all the revenue these  models are generating, and the downstream supply  

chain. China doesn't have that yet. They  have not built the scale of infrastructure  

to invest in models, get to the capabilities,  and then deploy these models at such scale. 

When you look at Anthropic,  they're at $20 billion ARR. 

The margins are sub-50 percent, at least  as last reported by The Information. 

So that's $13 or $14 billion of compute that it's  running on rental cost-wise, which is actually $50  

billion worth of CapEx that someone laid out  for Anthropic to generate their current revenue. 

China has just not done this. If and when Anthropic 10Xs revenue again—and  

I think our answer would be when, not if—China  doesn't have the compute to deploy at that scale. 

So there is some sense that  we're in a fast takeoff. 

It's not like we're talking  about a Dyson sphere by X date,  

it's more like the revenue is compounding at  such a rate that it does affect economic growth. 

The resources these labs are  gathering are growing so fast. 

China hasn't done that yet, so in that case,  the US and the West are actually diverging. 

The flip side is that these infrastructure  investments have middling returns. 

Maybe they're not as good as hoped. Maybe Google is wrong for wanting  

to take free cash flow to zero and  spend $300 billion on CapEx next year. 

Maybe they’re just wrong and people on  Wall Street who are bearish and people  

who don't understand AI are correct. In that case, the US is building all  

this capacity but doesn't get great returns. Meanwhile, China is able to build a fully  

vertical, indigenized supply chain, instead of  the US/Japan/Korea/Taiwan/SE Asia/Europe countries  

together building this less vertical supply chain. In a sense, at some point China is able to scale  

past us if AI takes longer to get to certain  capability levels than the vast majority of  

your guests on this podcast believe. It's fast timelines, the US wins;  

long timelines, China wins. Yeah but I don't know what fast timelines means. 

I don't think you have to believe in AGI  to have the timelines where the US wins. 

Let's go back to memory. I think people on  Wall Street and people in the industry are  

understanding how big this is, but maybe generally  people don't understand what a big deal it is. 

So we've got this memory crunch,  as you were talking about. 

And earlier I was asking about,  oh, could we solve for the EUV  

tool shortage by going back to seven nanometers? So let me ask a similar question about memory. 

HBM is made of DRAM, but has three  to four times fewer bits per wafer  

area than the DRAM it's made out of. Is it possible that accelerators in the  

future could just use commodity  DRAM and not HBM, so we can get  

much more capacity out of the DRAM we have? The reason I think this might be possible is,  

if we're going to have agents that are  just going off and doing work, and it's  

not a synchronous chatbot application, then you  don't necessarily need extremely fast latency. 

Maybe you can have lower bandwidth,  because the reason you stack DRAM into  

HBM is for higher bandwidth. Is it possible to go to HBM  

accelerators and basically have the opposite  of Claude Code Fast, like have Claude Slow? 

At the end of the day, the incremental  purchaser who's willing to pay the highest  

price for tokens also ends up being  the one that's less price-sensitive. 

Compute should be allocated, in a capitalistic  society, towards the goods that have the  

highest value, and the private market  determines this by willingness to pay. 

To some extent, Anthropic could  actually release a slow mode. 

They could release Claude Slow Mode and increase  tokens per dollar by a significant amount. 

They could probably reduce the price of Opus 4.6  by 4-5x and reduce the speed by maybe just 2x. 

The curve on inference throughput versus  speed is already there just on HBM. 

And yet they don't, because no one  actually wants to use a slow model. 

Furthermore, on these agentic tasks, it's great  that the model can run at a time horizon of hours. 

But if the model was running slower,  those hours would become a day. 

Vice versa, if the model is running  faster, those hours become an hour. 

No one really wants to move to a day-long wait  period, because the highest-value tasks also have  

some time sensitivity to them. I struggle to see… Yes,  

you could use regular DRAM. There are a couple of challenges with this. 

One of the core constraints of chips is  that a chip is a certain size, and all  

of the I/O escapes on the edges. Often, the left and right of the  

chip are HBM—so the I/O from the chip  to the HBM is on the sides—and then the  

top and bottom are I/O to other chips. If you were to change from HBM to DDR,  

all of a sudden this I/O on the edge  would have significantly less bandwidth,  

but significantly more capacity per chip. But the metric you actually care about  

is bandwidth per wafer, not bits per wafer. Because the thing that is constraining the FLOPS  

is just getting in and out the next matrix,  and for that you just need more bandwidth. 

Yeah, getting out the weights and  getting in and out the KV cache. 

In many cases, these GPUs are not  running at full memory capacity. 

It's obviously a system design thing:  model, hardware, and software co-design. 

You have to figure out how much KV cache  you need, how much you keep on the chip,  

how much you offload to other chips and  call when you need it for tool calling,  

and how many chips you parallelize this on. Obviously, the search space for this is very  

broad, which is why we have InferenceX,  an open-source model that searches all  

the optimal points on inference for a  variety of different chips and models. 

The point is, you're not always  necessarily constrained by memory capacity. 

You can be constrained by FLOPS, network  bandwidth, memory bandwidth, or memory capacity. 

If you really simplify it down,  there are four constraints,  

and each of these can break out into more. If you switch to DDR, yes, you produce  

four times the bits per DRAM wafer, but all of  a sudden the constraints shift a lot and your  

system design shifts. You go slower.  Is the market smaller? Maybe. But also,  

all these FLOPS are wasted because they're  just sitting there waiting for memory. 

You don't need all that capacity because you can't  really increase batch size because then the KV  

cache would take even longer to read. Makes sense. What is the bandwidth  

difference between HBM and normal DRAM? An HBM4 stack—let's talk about the stuff  

that's in Rubin, because that's what we've been  indexing on—is 2048 bits across, connected in an  

area that's 13 millimeters wide. It transfers memory at around  

10 giga-transfers a second. So a stack of HBM4 is 2048 bits on  

an area that's roughly 11 to 13 millimeters wide. That's the shoreline you're taking on the chip. 

In that shoreline, you have 2048 bits  transferring at 10 giga-transfers per second. 

You multiply those together and divide by eight,  

bits to a byte, and you're at roughly  2.5 terabytes a second per HBM stack. 

When you look at DDR, in that same  area, it's maybe 64 or 128 bits wide. 

That DDR5 is transferring at anywhere from  6.4 to maybe 8,000 giga-transfers a second. 

So your bandwidth is significantly lower. It's 64 times 8,000 divided by  

eight, which puts you at 64 gigabytes a second. Even if you take a generous interpretation of  

128 times 8 giga-transfers, you're at 128  gigabytes a second for the same shoreline,  

versus 2.5 terabytes a second. There's an order of magnitude  

difference in bandwidth per edge area. If your chip is a square, or 26 by 33  

millimeters—which is the maximum size for an  individual die—you only have so much edge area. 

On the inside of that chip,  you put all your compute. 

There are things you can do to try and  change that, like more SRAM or more caching. 

But at the end of the day, you're  very constrained by bandwidth. 

Then there's the question of where you can  destroy demand to free up enough for AI. 

I guess the picture is especially bad because,  as you're saying, if it takes four times more  

wafer area to get the same byte, for HBM you have  to destroy four times as much consumer demand for  

laptops and phones to free up one byte for AI. What does this imply for the next year or two? 

Sorry for the run-on question, in your newsletter  you said 30% of Big Tech's CapEx in 2026 is going  

towards memory? Yes. 

That's insane, right? Of the $600 billion  or whatever, 30% is going just to memory. 

Yes. Obviously, there's some level  of margin stacking that Nvidia does,  

so you have to separate that out and apply  their margin to the memory and the logic. 

But at the end of the day, a third  of their CapEx is going to memory. 

That's crazy. What should we expect over the  next year or two as this memory crunch hits? 

The memory crunch will continue to get  harder, and prices will continue to go up. 

This affects different parts  of the market differently. 

Are people going to hate AI more and more? Yes, because smartphones and PCs are not  

going to get incrementally better year on year. In fact, they're going to get incrementally worse. 

If you look at the bill of materials for an  iPhone, what fraction of it is the memory? 

How much more expensive does an iPhone get  if the memory is two times more expensive? 

I believe an iPhone has 12 gigabytes of memory. Each gig used to cost roughly $3-4, so that's $50. 

But now the price of memory has tripled. Let's say it's $12 per gig for DDR. 

Now you're talking about $150 versus $50. That's a $100 increase in cost for Apple. 

Apple has some margin, they're  not just going to eat the margin. 

NAND also has the same market dynamics,  so in reality, it's probably a $150  

increase on the iPhone. So now that’s a $100 cost  

increase and that’s just on the DRAM. The NAND also has the same sort of market. 

So in fact it’s probably a  $150 increase on the iPhone. 

Apple either has to pass that  on to the consumer or eat it. 

I don't see Apple reducing their margin  too much, maybe they eat a little bit. 

But at the end of the day, that means the end  consumer is paying $250 more for an iPhone. 

Now that’s just on last  year’s pricing versus today’s. 

There is some lag before Apple feels the heat  because they tend to have long-term contracts  

for memory that last three months to a year. But at the end of the day, Apple gets hit  

pretty hard by this. They won't really  

adjust until the next iPhone release. But that's the high end of the market,  

which is only a few hundred million phones a year. Apple sells two or three hundred million  

phones annually. The bulk of the market is mid-range and low-end. 

It used to be that 1.4 billion  smartphones were sold a year. 

Now we're at about 1.1 billion. Our projections are that we might  

drop to 800 million this year, and  down to 500 or 600 million next year. 

We look at data points out of China  from some of our analysts in Asia,  

Singapore, Hong Kong, and Taiwan. They've been tracking this,  

and they see Xiaomi and Oppo cutting low-end  and mid-range smartphone volumes by half. 

Yes, it’s only a $150 BOM increase on a $1,000  iPhone where Apple has some larger margin. 

But for smaller phones, the percentage of the BOM  that goes to memory and storage is much larger. 

And the margins are lower, so there's  less capacity to even eat the margins. 

And they have also generally tended not  to do long-term agreements on memory. 

Why this is a big deal is that if smartphone  volumes halve, that drop will happen in  

the low and mid-range, not the high end. So it’s not like the bits released are halving. 

Currently, consumer devices account  for more than half of memory demand. 

Even if you halve smartphone volumes,  because of the shape of the halving,  

the low end gets cut by more than half, while  the high end gets cut by less than half,  

because you and I will still buy the high-end  phones that cost north of a thousand dollars. 

We'll buy them even if they get  a little bit more expensive. 

And Apple's volumes will not go down as  much as a low-end smartphone provider.  

The same applies to PCs. What this  does to the market is quite drastic. 

DRAM gets released and goes to AI chips, who are  willing to do longer-term contracts and pay higher  

margins, because at the end of the day the margin  they extract from the end user is much larger. 

This probably leads to people hating AI even more. Today, you already see all the memes on PC  

subreddits and gaming PC Twitter. It's cat dancing videos saying,  

"This is why memory prices have doubled and  you can't get a new gaming GPU or desktop." 

It's going to be even worse when memory  prices double again, especially DRAM. 

Another interesting dynamic is that  it's not just DRAM, it's also NAND. 

NAND is also going up in price. Both of these markets have expanded capacity very  

slowly over the last few years, NAND almost zero. The percentage of NAND that goes to phones and  

PCs is larger than the percentage  of DRAM that goes to phones and PCs. 

As you destroy demand, mostly for  DRAM purposes, you unlock more NAND  

that gets allocated and can go to other markets. The price increases of DRAM will be larger than  

those of NAND because you've released  more from the consumer, and in fact,  

you've produced more memory for AI. Sorry, maybe you just explained  

it and I missed it. Is it because SSDs are  

being used in large quantities for data centers? They are, but not in as large quantities as DRAM. 

Okay, so they will also increase because  they'll be using some quantity, but there's  

not as much of a need as there is for HBM. Makes  sense. One thing I didn't appreciate until I was  

reading some of your newsletters is that the  same constraints preventing logic scaling over  

the next few years are quite similar to what's  preventing us from producing more memory wafers. 

In fact, literally the same exact machine,  this EUV tool, is needed for memory. 

So I guess the question someone could ask right  now is, why can't we just make more memory? 

The constraints, as I was mentioning earlier,  are not necessarily EUV tools today or next year. 

They become that as we get to  the latter part of the decade. 

Currently, the constraints are more that  they physically just haven't built fabs. 

Over the last three to four years,  these vendors have not built new fabs  

because memory prices were really low. Their margins were low, and in fact,  

they were losing money in 2023 on memory. So they decided they weren't building new fabs. 

The market slowly recovered over time but  never really got amazing until last year. 

In 2024, we were banging on the drums  that reasoning means long context,  

which means a large KV cache, which  means you need a lot of memory demand. 

We've been talking about that  for a year and a half, two years. 

People who understand AI went  really long on memory then. 

So you’ve seen that dynamic, but now  it has finally played out in pricing. 

It took so long for what was  obvious: long context means the KV  

cache gets bigger, you need more memory. Half the cost of accelerators is memory. 

Of course they're going to  start going crazy on it. 

It took a year for that to  actually reflect in memory prices. 

Once memory prices reflected that, it  took another three to six months for the  

memory vendors to start building fabs. Those fabs take two years to build. 

So we won't have really meaningful fabs to even  put these tools in until late 2027 or 2028. 

Instead, you've seen some really  crazy stuff to get capacity. 

Micron bought a fab from a company in  Taiwan that makes lagging-edge chips. 

Hynix and Samsung are doing some pretty  crazy things to try and expand capacity  

at their existing fabs, which also have  large knock-on effects in the economy. 

So why can't we build more capacity? There's nowhere to put the tools. 

It's not just EUV; there are other  tools involved in DRAM and logic. 

In logic, for N3, about 28%  

of the cost of the final wafer is EUV. When you look at DRAM, it's in the teens. 

It's going up, but it's a much  smaller percentage of the cost. 

These other tools are also bottlenecks, although  their supply chains are not as complex as ASML's. 

You see Applied Materials, Lam  Research, and all these other  

companies expanding capacity a lot as well. But you don't have anywhere to put the tool,  

because the most complex buildings people make  are fabs, and fabs take two years to build. 

I interviewed Elon recently, and his whole plan  is that they're going to build this TeraFab  

and they're going to build the clean rooms. I won't even ask you about the dirty rooms thing,  

but let's say they build the clean rooms. I have a couple of questions. 

One, do you think this is the kind of  thing that Elon Co. could build much  

faster than people conventionally build it? This is not about building the end tools. 

This is just about building the facility itself. How complicated is it to just build  

the clean room extremely fast? Is this something that Elon, with his "move  

fast" approach, could do much faster if that's  what we're bottlenecked on this year or next year? 

Two, does that even matter if, in two years,  your view is that we're not bottlenecked on  

clean room space, but on the tooling? As with any complex supply chain,  

it takes time, and constraints shift over time. Even if something is no longer a constraint, that  

doesn't mean that market no longer has margin. For example, energy will not be a big bottleneck  

a couple of years from now, but that  doesn't mean energy isn't growing super  

fast and there's no margin there. It's just not the key bottleneck. 

In the space of fabs, clean rooms are the  biggest bottleneck this year and next year. 

As we get to 2028, 2029, 2030, there  will still be constraints there. 

The thing about Elon is he has a tremendous  capability to garner physical resources and  

really smart people to build things. The way he recruits amazing people  

is by trying to build the craziest stuff. In the case of AI, that hasn't really worked  

because everyone's trying to build AGI. Everyone  is very ambitious. But in the case of going to  

Mars, making rockets that land themselves, fully  autonomous electric cars, or humanoid robots,  

these are methods of recruiting the people who  think that's the most important problem in the  

world to work on that problem, because  he's the only one trying really hard. 

In the case of semiconductors, he stated he wants  to make a fab that's a million wafers per month. 

No one has a fab that big. It's possible that he's able to recruit a  

lot of really awesome people and get them on this  crazy task of building a million wafers a month. 

Step one is to build the clean room,  and that I think he probably can do. 

His mindset around deleting things, that it  can be dirty, it's fine, is probably not right. 

Actually I think it’s 100% not right. You need the fab to be very clean. 

All of the air in the fab gets replaced  every three seconds, it’s that fast. 

There have to be so few particles. But I think he can build the clean room. 

It'll take a year or two. Initially, it won't be super fast,  

but over time, he'll get faster at it. The really complex part is actually developing  

a process technology and building wafers. I don't think he can develop that quickly. 

That has a lot of built-up knowledge. The most complicated integration of  

very expensive tools and supply chains  is done by TSMC, Intel, or Samsung. 

These two other companies aren't even that  great at it, and they're tremendously complex. 

How surprised would you be if in 2030  there just happened to be some total  

disruption where we're not using EUV? What if we're using something that has  

much better effects, is much simpler to produce,  and can be produced in much bigger quantities? 

I'm sure as an industry insider that  sounds like a totally naive question,  

but do you see what I'm asking? What probability should we put on  

something coming totally out of left  field to make all of this irrelevant? 

Something that's very simple and easy to  scale, I assign a very, very low probability. 

There are a number of companies  working on effectively particle  

accelerators or synchrotrons that generate  light that's either 13.5 nanometer, like EUV,  

or an even narrower wavelength, like X-ray at  7 nanometers, to then use in lithography tools. 

But those things are massive particle  accelerators generating this light. 

It's a very complicated thing to build. There are a couple of companies and I think  

that could be a big disruption  to the industry beyond EUV. 

But I don't think we're going to  magically build something new that  

is direct write and super simple, and can  be manufactured at huge volumes, although  

there are some attempts to do things like this. I ask because if you think about Elon's companies  

in the past, rocketry was this thing that was  thought to be—and is—incredibly complicated. 

Look, I'm just a naive yapper compared to Elon.  What have I built? So maybe it's possible. 

In order to build more memory in the  future, could we build 3D DRAM the way  

we do 3D NAND and then go back to DUV? That is the hope currently. Everyone's  

roadmap for 3D DRAM is that you'll still use EUV  because you want to have that tighter overlay. 

When you're doing these subsequent processing  steps, everything is vertically stacked and you  

have more layers on top of each other. You want the pitches to be tighter. 

So generally, people are still  trying to do it with EUV. 

But what 3D would do is change the calculation  of how many bits a single EUV pass can make. 

That number would go up drastically if you  go to 3D DRAM. That is the hope. Right now,  

everyone's roadmap goes from the current 6F cell,  to a 4F cell, and then finally 3D DRAM by the end  

of the decade or early next decade. There's still a lot of R&D,  

manufacturing, and integration to be done. I wouldn't call that out of the cards. 

I think it's very likely going to happen. It's also going to require a huge  

retooling of fabs. The breakdown of  

tools in a fab will be very different. The lithography tool is actually the  

only thing that isn't that different. But the number of them relative to different  

types of chemical vapor deposition, atomic layer  deposition, dry etch, or different kinds of etch  

chambers with different chemistries… You have all  these different tools for different process nodes. 

You can't just convert a logic fab to a  DRAM fab, or vice versa, or a NAND fab  

to a DRAM fab, in a short amount of time. In the same way, existing DRAM fabs require a  

lot of retooling just to go from 1-alpha to 1-beta  to 1-gamma process nodes, because they have to  

add DUV and change the chemistry stacks for when  you’re using EUV in terms of deposition and etch. 

And the EUV tool has to be there. Furthermore, when you change to 3D DRAM,  

there's going to be an even larger shift, so a  lot of retooling of these fabs needs to happen. 

That would be a big disruption. That would make EUV demand generally lower. 

But as we've seen across time, lithography demand  as a percentage of wafer cost has trended up. 

Around the 2014 era, it was 17% of the wafer cost,  and it's gone to 30% over the last fifteen years. 

For DRAM, it was in the low to mid-teens,  and now it's trended toward the high teens. 

Before we get to 3D DRAM, it'll  likely cross into the 20% range. 

But then, if we get to 3D DRAM, the total end  wafer cost as a percentage of EUV tanks again. 

I guess you care less about the percent of cost  and more about how much it bottlenecks production. 

Right, but the percentage of cost— It’s a proxy, yeah. If you're Jensen  

or Sam Altman, or whoever stands to  gain a lot from scaling up AI compute,  

there are these stories that they'd go to  TSMC and say, "Why can't we access Y and Z?" 

But I think the point you're  making is that it doesn't really  

matter what TSMC does in some sense. In fact, even if you have Intel and  

Samsung building more foundries, in the  long run, you're going to be bottlenecked  

by ASML and other tool and material makers. First, is that a correct interpretation? 

Second, should Silicon Valley people be  going to the Netherlands right now to try  

to pitch ASML to make more tools so that  in 2030 they can have more AI compute? 

It's a funny dynamic we saw  in 2023, 2024, and 2025. 

People who saw the energy bottleneck  before others asymmetrically went to  

Siemens, Mitsubishi, and of course GE  Vernova, and bought up turbine capacity. 

Now they're able to charge  excess amounts for deploying  

these turbines in places because of energy. In the same sense, this could be done for EUV,  

except ASML is not just going to trust any  random bozo who wants to buy EUV tools. 

These turbines are much cheaper than EUV  tools, and there's many more of them produced. 

Especially once you get to industrial gas  turbines, not just combined-cycle but the cheaper,  

smaller, less efficient ones, people put  down deposits for these. Someone could  

do this. Someone should go to the Netherlands  and be like, "I'll pay you a billion dollars. 

You give me the right to purchase ten EUV tools  two years from now, and I'm first in line." 

Then over those two years, you go around  and wait for everyone to realize, "Oh crap,  

I don't have enough EUV tools," and you  try to sell your option at some premium. 

All you're effectively doing  is saying, "ASML, you're dumb. 

You weren't making enough margin on these. I'm going to make a margin." 

The question is, will ASML even  agree to this? I don't think so. 

There's a world where they at least get the  demand signal from that to increase production. 

Potentially. I agree. But it sounds like you're  

saying they couldn't even increase production  if they wanted to, given the supply chain. 

Right. But that's exactly the market in  which… If they can't increase production,  

just like TSMC cannot increase production  that fast, and yet demand is mooning,  

then the obvious solution is to arbitrage this. You and I know demand is way higher than they're  

projecting and their capability to build. You arbitrage this by locking up the capacity,  

doing a forward contract, and then  trying to sell it at a later date  

once other people realize everything is  fucked and we don't have enough capacity. 

Then you'll have this insane margin that  ASML and TSMC should have been charging. 

But the thing is, I don't know if  ASML and TSMC will ever agree to this. 

Let me ask you about power now. It sounds like you think power  

can be arbitrarily scaled. Not arbitrarily, but yes. 

But beyond these numbers. If I'm  remembering correctly, your blog post on  

how AI labs are increasing power implied that  GE Vernova, Mitsubishi, and Siemens could  

produce 60 gigawatts a year in gas turbines. Then there are these other sources,  

but they're less significant than the turbines. Only a fraction of that goes to AI, I assume. 

If in 2030 we have enough logic and memory to  do 200 gigawatts a year, do you just think that  

these things are on a path to ramp up to more  than 200 gigawatts a year, or what do you see? 

Right now we're at 20 or 30. This is critical IT capacity, by the way,  

which is an important thing to mention. When I'm talking about these gigawatts,  

I'm talking about critical IT capacity. Server plugged in, that's how much power it pulls. 

But there are losses along the chain. There is loss on transmission,  

conversion, cooling, et cetera. So you should gross this factor  

up from 20 gigawatts for this year, or 200  gigawatts by the end of the decade, to some  

number 20-30% higher. Then you have capacity  factors. Turbines don't run at 100 percent. 

If you look at PJM, which I think is the largest  grid in America—covering the Midwest and some of  

the Northeast area—in their models they want  to have roughly 20 percent excess capacity. 

Within that 20 percent excess capacity,  they're running all the turbines at 90%  

because they are derated some for  reliability, maintenance, and so on. 

In reality, the nameplate capacity for energy is  always way higher than the actual end critical IT  

capacity because of all these factors. But it's  not just turbines. If you were just making power  

from turbines, that's simple, boring, and easy. Humans and capitalism are far more effective. 

The whole point of that blog was that, yes, there  are only three people making combined-cycle gas  

turbines, but there's so much more we can  do. We can do aeroderivatives. We can take  

airplane engines and turn them into turbines. There are even new entrants in the market,  

like Boom Supersonic trying to  do that and working with Crusoe. 

Also there's all the other ones like  that already exist in the market. 

There are also medium-speed  reciprocating engines: engines  

that spin in circles, like a diesel engine. There are ten people who make engines that way. 

I'm from Georgia, and people used  to be like, "Oh man, you got a  

Cummins engine in there," regarding RAM trucks. Automobile manufacturing is going down, so these  

companies all have capacity and could scale  and convert that for data center power. 

You stick all these reciprocating engines in. It's not as clean as combined-cycle, but maybe you  

can convert them from diesel to gas if you want.  What about ship engines? All of these engines for  

massive cargo ships are great. Nebius is doing that for  

a Microsoft data center in New Jersey. They're running ship engines to generate power. 

Bloom Energy is doing fuel cells. We've been very positive on them for  

a year and a half now because they have such  a capability to increase their production. 

Their payback period for a production  increase is very fast, even if the cost  

is a little bit higher than combined-cycle,  which is the best for cost and efficiency. 

Then there's solar plus battery, which can come  online as those cost curves continue to come down. 

There's wind, where you might only expect 15  percent of the maximum power because things  

oscillate, but you add batteries. There are  all these things. The other thing is that the  

grid is scaled so we don't cut off power at  peak usage on the hottest day of the summer. 

But in reality, that's a load spike  that is 10-20% higher than the average. 

If you just put enough utility-scale  batteries, or peaker plants that only  

run a small portion of the year—and those could  be gas, industrial gas turbines, combined-cycle,  

batteries, or any of the other sources  I mentioned—then all of a sudden you've  

unlocked 20% of the US grid for data centers. Most of the time that capacity is sitting idle. 

It's really only there for that peak, which is  just a few hours over a few days of the year. 

If you have enough capacity  to absorb that peak load,  

then all of the sudden you’ve transferred it all. Today, data centers are only 3-4% of the power of  

the US grid, and by 2028 they'll be 10%. But if you can unlock 20% of the US grid  

like this, it's not that crazy. The US grid is terawatt-level,  

not hundreds-of-gigawatts-level. So we can add a lot more energy. I'm not saying  

it's easy. These things are going to be hard. There's a lot of hard engineering,  

risks people have to take, and new  technologies people have to use. 

But Elon was the first to do this behind-the-meter  gas, and since then we've seen an explosion of  

different things people are doing to get power. They're not easy,  

but people are gonna be able to do them. The supply chains are just way simpler than chips. 

Interesting. He made the point during the  interview that for the specific blade for  

the specific turbine he was looking at, the lead  times go out beyond 2030. Your point is that— 

That's great. There are so many other ways to  make energy. Just be inefficient. It's fine. 

Right now, combined-cycle gas turbines  have CapEx of $1,500 per kilowatt. 

Are you saying it would make sense to  have either technologies that are much  

more expensive than that, or other things are  getting cheap enough to make it competitive? 

Exactly. It can be as high as $3,500 per kilowatt. It could be twice as much as the cost of  

combined-cycle, and the total cost of the GPU on  a TCO basis has only gone up a few cents per hour. 

Because we've been talking about Hopper pricing,  $1.40, let's say the power price doubles. 

The Hopper that was $1.40 is now $1.50 in cost. I don't care, because the models are improving so  

fast that the marginal utility of them is worth  way more than that ten-cent increase in energy. 

So you're saying 20 percent of the grid—the grid  is about one terawatt—can just come online from  

utility-scale batteries, increasing what  you'd be comfortable putting on the grid. 

The regulatory mechanism  there is not easy, by the way. 

But that's 200 gigawatts, if  that hypothetically happens. 

Just from the different sources of gas generation  you mentioned—the different kinds of engines  

and turbines—combined, how many gigawatts  could they unlock by the end of the decade? 

We're tracking this in our data. There are over 16 different manufacturers  

of power-generating things just from gas alone. Yes, there are only three turbine manufacturers  

for combined-cycle, but we're  tracking 16 different vendors,  

and we have all of their orders. It turns out there are hundreds of  

gigawatts of orders to various data centers. As we get to the end of the decade,  

we think something like half of the capacity  that's being added will be behind the meter. 

Behind the meter is almost always more expensive  than grid-connected, but there are just a lot of  

problems with getting grid-connected: permits and  interconnection queues and all this sort of stuff. 

So even though it's more expensive,  people are doing behind the meter. 

What they're doing behind the meter ranges widely. It could be reciprocating engines, ship engines,  

or aeroderivatives. It could be combined-cycle,  

although combined-cycle is not  that great for behind the meter. 

It could be Bloom Energy fuel  cells, or solar plus battery. 

It could be any of these things. And you're saying any of these  

individually could do tens of gigawatts? Any of these individually will do tens of  

gigawatts, and as a whole, they  will do hundreds of gigawatts. 

Okay. So that alone should more than— Electrician wages will probably  

double or triple again. There are going to be a lot of new people entering  

that field, and a ton of people who make money,  but I don't see that as the main bottleneck. 

Right now in Abilene, at the 1.2-gigawatt data  center that Crusoe is building for OpenAI,  

I think they have 5,000 people  working there, or at peak they did. 

If you turn that into 100 gigawatts—and  I'm sure things will get more efficient  

over time—that would be 400,000 people  it would take to build 100 gigawatts. 

If you think about the US labor force, and  how many electricians there are and how  

many construction workers there are… I  guess there are 800,000 electricians. 

I don't know if they're all  substitutable in this way. 

There are millions of construction workers. But if we're in a world where we're adding  

200 gigawatts a year, are we going to be  crunched on labor eventually, or do you  

think that is actually not a real constraint? Labor is a big constraint. It's a humongous  

constraint in this. People have to  be trained. Likewise, we'll probably  

start importing the highest-skilled labor. It makes sense that a really high-skilled  

electrician in Europe who was working  on destroying power plants now comes  

to America and is building high-voltage  electricity moving across a data center. 

Humanoid robots or robotics at least might start  to help, but the main factor for reducing the  

number of people is going to be modularizing  things and making them in factories in Asia. 

Unfortunately for America, places like Korea,  Southeast Asia, and in many ways China as well  

are going to ship more and more built-out sections  of the data center and those will be shipped in. 

Today you currently ship servers or a rack in,  and then you plug that into different pieces that  

you're shipping from different places. But now you'll ship it to a factory  

and integrate the entire thing. Maybe this is a two-megawatt block,  

and this block goes from high-voltage AC  power to the DC voltage that you deliver  

to the rack, or something like this. Or with cooling, you ship a fully  

integrated unit that has a lot of the  cooling subsystems already put together,  

because plumbers are also a big constraint here. Furthermore, instead of just a single rack where  

you have people wiring up all these racks with  electricity, you take a skid and put an entire  

row of servers on it that is  shipped directly from the factories. 

Today, a single rack may be 120 or 140 kilowatts,  but as we get to next-generation Nvidia Kyber and  

things like that, it's almost a megawatt. In addition, if you do an entire  

row, it'll have the rack, the networking, the  cooling, and the power all integrated together. 

Now when you come in, you have much less to cable. There's less networking fiber, fewer power  

connections, and fewer plumbing things. This can drastically reduce the number  

of people working in data centers, so our  capability to build them will be much larger. 

Along the way, some people will move faster  to new things, and some will move slower. 

Crusoe and Google have been talking  a lot about this modularization,  

as have companies like Meta and many others. The people who move faster to new things may  

face delays, while the people who  are slower will face labor problems. 

There will always be dislocations in the market  because this is a very complex supply chain. 

At the end of the day, it's still  simple enough that we will be able  

to solve it through capitalism and human  ingenuity on the timescales required. 

Speaking of big problems to solve, Elon  Musk is very bullish on space GPUs. 

If you're right that power is not a constraint  on Earth… I guess the other reason they would  

make sense is that even if there will be  enough gas turbines or whatever on Earth,  

Elon's next argument is that you can't get the  permitting to build hundreds of gigawatts on  

Earth. Do you buy that argument? Land-wise, America is big. Data  

centers don't actually take up that  much space, so you can solve that. 

Permitting-wise, air pollution permits are  a challenge, but the Trump administration  

made it much easier. You go to Texas,  

and you can skip a lot of this red tape. Elon had to deal with a lot of this complex  

stuff in Memphis, and then building a power  plant across the border for Colossus 1 and 2. 

But at the end of the day, there's a lot more  you can get away with in the middle of Texas. 

Given that Elon lives in Texas,  why didn't he just go to Texas? 

I think it was partially that they over-indexed  on grid power for a temporary period of time. 

That's just what they thought they needed more of. Because they had an aluminum refinery  

connected to the grid there. It was actually an idled appliance factory. 

But I think they may have indexed more to  grid power, water access, and gas access. 

I think they bought that knowing the gas  line was right there and they were going  

to tap it. Same with water. It was a  whole host of different constraints. 

It was probably an area where  electricians were easier to find. 

At the end of the day, I'm not  exactly sure why they chose that site. 

I bet Elon would've chosen somewhere in  Texas if he could've gone back because  

of the regulatory challenges he faced. Ultimately, permitting is a challenge,  

but America is a big place with 50  states, and things will get done. 

There are a lot of small jurisdictions where  you can just transport in all the workers  

you need for a temporary period of three to  twelve months, depending on the contractor. 

You can put them in temporary housing and pay out  the butt, because labor is very cheap relative  

to the GPUs and the networking, and the end  value of the tokens it's going to produce. 

So there is plenty of room to  pay for all of these things. 

Also, people are also diversifying now. Australia, Malaysia, Indonesia, and India  

are all places where data centers  are going up at a much faster pace. 

But currently, over 70% of AI  data centers are still in America,  

and that continues to be the trend. People are figuring out how to build these things. 

Ultimately, dealing with permitting and  red tape in middle-of-nowhere Texas,  

Wyoming, or New Mexico is probably a hell of  a lot easier than sending stuff into space. 

Other than the economic argument making less sense  once you consider that energy is a small fraction  

of the total cost of ownership of a data center,  what are the other reasons you're skeptical? 

Obviously, power is basically free in space. That's the reason to do it. 

Yeah, that's the reason to do it. But there are all the other counterarguments. 

Even if power costs double on Earth, it's  still a fraction of the total cost of the GPU. 

The main challenge is… We have  ClusterMAX, which rates all the neoclouds. 

We test over 40 cloud companies,  including the hyperscalers and neoclouds. 

Outside of software, what differentiates these  clouds the most is their ability to deploy and  

manage failure. GPUs are horrendously unreliable.  Even today, around 15% of Blackwells that get  

deployed have to be RMA'd. You have to take them out. 

Sometimes you just have to plug them  back in, but sometimes you have to take  

them out and ship them back to Nvidia or  their partners who do the RMAs and such. 

What do you make of Elon's argument that after an  initial phase, they actually don't fail that much? 

Sure, but now you've done this, tested them all,  deconstructed them, put them on a spaceship,  

launched them into space, and then put  them online again. That takes months. If  

your argument is that a GPU has a useful life of  five years, and this takes six additional months,  

that is 10% of your cluster's useful life. Because we're so capacity-constrained,  

that compute is theoretically most valuable  in the first six months you have it. 

We're more constrained now  than we will be in the future. 

That compute can contribute to a better  model in the future, or generate revenue  

today that you can use to raise more money. All these things make now the most important  

moment, but you've potentially delayed  your compute deployment by six months. 

What separates these cloud providers is… We  see some clouds taking six months to deploy  

GPUs right here on Earth. We see clouds that take  

a lot less than six months. So the question is, where does space get in there? 

I don't see how you could test them all on Earth,  deconstruct them, and ship them to space without  

it taking significantly longer than just leaving  them in the facility where you tested them. 

The question I wanted to ask is about  the topology of space communication. 

Right now, Starlink satellites talk to  each other at 100 gigabits per second. 

You could imagine that being much  higher with optical intersatellite  

laser links optimized for this. That actually ends up being quite  

close to InfiniBand bandwidth,  which is 400 gigabytes a second. 

But that's per GPU, not per rack. So multiply  that by 72. Also, that was Hopper. When you go  

to Blackwell and Rubin, that 2x's and 2x's again. But how much compute is happening per… During  

inference, are the different scale-ups  still working together, or is inference just  

happening as a batch within a single scale-up? A lot of models fit within one scale-up domain,  

but many times you split them  across multiple scale-up domains. 

As models become more and more sparse,  which is the general trend, you want to  

ping just a couple of experts per GPU. If leading models today have hundreds,  

if not a thousand, of experts, then you'd want to  run this across hundreds or thousands of chips,  

even as we advance into the future. So then you end up with the problem of  

needing to connect all these satellites  together for communications as well. 

That would be tough. If there's a world where  you could do inference for a batch on a single  

scale-up, then maybe it's more plausible. But if not, it's a different story. 

Networking these chips together  is a problem, and you can't just  

make the satellite infinitely large. There are a lot of physics challenges to  

making a satellite really big. That's why you need these  

interconnects between the satellites. Those  interconnects are more expensive. In a cluster,  

15-20% of the cost is networking. All of a sudden, you're using space lasers  

instead of simple lasers that are manufactured in  volumes of millions with pluggable transceivers. 

And those things are very unreliable as well,  more unreliable than the GPUs by the way. 

Across the life of a cluster, you have  to unplug and clean them all the time. 

You have to unplug and replug  them just for random reasons. 

These things are just not as reliable. So you've got that problem as well. 

You've got a more expensive, complicated  space laser to communicate instead of this  

pluggable optical transceiver that's  been produced in super high volume. 

So all in all, what does that  imply for space data centers? 

Space data centers effectively are  not limited by their energy advantage. 

They are limited by the same contended resource. We can only make two hundred gigawatts  

of chips a year by the end of the decade. What are we going to do to get that capacity? 

It doesn't matter if it's on land or in space. It doesn’t really matter,  

because you can build that power. Human capabilities and capacity could get  

to the period where we're adding a terawatt  a year globally of various types of power. 

At some point, we do cross the chasm where space  data centers make sense, but it's not this decade. 

It is much further out, once energy  constraints actually become a big bottleneck  

and land permitting becomes a much bigger  bottleneck as it subsumes more of the economy. 

And crucially, once chips  are no longer the bottleneck. 

Right now, chips are the biggest bottleneck. You want them deployed and working on  

AI the moment they're manufactured. There are a lot of things people are  

doing to increase that speed faster and faster. They’re modularizing data centers, or even  

modularizing racks where you put the chip in at  the data center, but only the chip and everything  

else is already wired up and ready to go. There are things like this people are doing to  

decrease that time that you cannot do in space. At the end of the day, all that matters in a  

chip-constrained world is getting  these chips producing tokens ASAP. 

Maybe by 2035, the semiconductor industry,  ASML, Zeiss, and suppliers like Lam  

Research and Applied Materials and other fab  manufacturers will catch up once the pendulum  

swings and we are able to make enough chips. Then we will be optimizing every dial and it makes  

sense to optimize the 10-15% of energy costs. As we move to ASICs potentially,  

and if Nvidia's margins aren't +70%, maybe  that energy cost becomes 30% of the cluster. 

These are the things to optimize. But Elon doesn't win by doing 20% gains. He  

never wins that way. Elon wins when he swings for  the fences and does 10X gains. That's what SpaceX  

is about. That's what Tesla is about. All of his  success has been about that, not chasing the 20%. 

I think space data centers will eventually  be a 10X gain as Earth's resources get more  

and more contentious, but that's not this decade. Just to drive some intuition about how much land  

there is on Earth… Obviously, for the chips  themselves, especially if you move to a world  

where you have racks that have megawatts— That's the other thing. If manufacturing is  

the constraint, right now it's roughly one  watt per square millimeter for AI chips. 

One easy way to improve that is to pump  it to two watts per square millimeter. 

You may not get 2x the performance,  you may only get 20% more performance,  

and that requires much more exotic cooling. It requires more complicated cold plates  

and complex liquid cooling, or maybe  even things like immersion cooling. 

In space, higher watts per  millimeter is very difficult,  

whereas on Earth, these are solved problems. One of these things enables you to get a lot  

more tokens, maybe 20% more tokens per wafer  that's manufactured, and that's a humongous win. 

Square millimeter, you mean of die area? Yeah, of die area. 

It would be better for space because more watts  per millimeter means the chip runs hotter. 

I guess this is a question of computer  chip engineering, but it cools to the  

fourth power by the Stefan-Boltzmann law. If you can run a very hot  

chip, it allows a lot of— No, you can't run it hotter. 

You can only run it denser. The problem is that getting  

the heat out of that dense area means you have to  move away from standard air and liquid cooling to  

more exotic forms of liquid cooling, or even  immersion, to get to higher power densities. 

That's more difficult in  space than it is on Earth. 

Maybe it's worth explaining at this point what  exactly a scale-up is and what it looks like for  

Nvidia versus Trainium versus TPUs. Earlier I was mentioning how  

communication within a chip is super fast. Communication within chips that are in the  

same rack is fast, but not as fast. It's on the order of terabytes. 

Communication very far away is on  the order of hundreds of gigabytes. 

As you get further distance, maybe  across the country, the order  

of magnitude is on the order of gigabytes. A scale-up domain is this tight domain  

where the chips are communicating  on the order of terabytes a second. 

For Nvidia, previously this meant  an H100 server had eight GPUs,  

and those eight GPUs could talk to  each other at terabytes a second. 

With Blackwell NVL72, they  implemented rack-scale scale-up. 

That meant all seventy-two GPUs in the rack could  connect to each other at terabytes a second. 

The speed doubled generation on generation, but  the most important innovation was going from eight  

to seventy-two in the domain. When we look at Google,  

their scale-up domain is completely different. It has always been on the order of thousands. 

With TPU v4, they had pods the  size of four thousand chips. 

With v8 or v7, they have pods in  the eight or nine thousand range. 

What's relevant here is that it's not the  same as Nvidia. It's not like for like.  

Google has a topology that's a torus. Every chip connects to six neighbors.  

Nvidia's 72 GPUs connect all-to-all.  They can send terabytes a second to  

any arbitrary other chip in that pod of scale-up. Whereas Google, you have to bounce through chips. 

If TPU 1 needs to talk to TPU 76, it has to bounce  through various chips, and there is always some  

blocking of resources when you do that because  that one TPU is only connected to six other TPUs. 

So there is a difference  in topology and bandwidth,  

and there are trade-offs and advantages to both. Google gets to have a massive scale-up domain,  

but they have the trade-off of bouncing  across chips to get from one to another. 

You can only talk to six direct neighbors. Amazon has mutated their scale-up domain. 

They're somewhere in between Nvidia and Google. They're trying to make larger scale-up domains. 

They try to do all-to-all to some extent with  switches, which is what Nvidia does, but they also  

use torus topologies like Google to some extent. As we advance forward to next generations,  

all three of them are moving more  towards a dragonfly topology. 

That means there are some fully connected elements  and some elements that are not fully connected. 

You can get the scale-up to be hundreds or  thousands of chips, but also have it not contend  

for resources when bouncing through chips. Related question: I heard somebody make the  

claim that the reason parameter scaling has been  slow—and only now are we getting bigger models  

from OpenAI and Anthropic—is that… The original  GPT-4 is over a trillion parameters, and only now  

are models starting to approach that again. I heard a theory that the reason is that  

Nvidia's scale-ups have just not  had that much memory capacity. 

Let's say you have a 5T model running at  FP8, so that's five trillion gigabytes. 

And then you have the KV cache, let's say it's— Just call it the same size. 

Okay, let's say it's the same size for one batch. So you need ten terabytes to be able to run… 

A single forward pass, yeah. And then only with the GB200 and NVL72  

do you have an Nvidia scale-up that has twenty  terabytes, and before that they were much smaller. 

Whereas Google, on the other hand, has had  these huge TPU pods that are not all-to-all,  

but still have hundreds of terabytes  of capacity in a single scale-up. 

Does that explain why parameter  scaling has been slow? 

I think it's partially the capacity and  bandwidth, but also as you build a larger  

model, the ability to deploy it is slower. In terms of what the inference speed is for  

the end user, that's kind of irrelevant. What's  really relevant is RL. What we've seen with these  

models and allocation of compute at a lab… There  are a few main ways you can allocate compute. 

You can allocate it to inference, i.e. revenue. You can allocate it to development,  

i.e. making the next model. You can allocate it to research. 

In development specifically, you  split it between pre-training and RL. 

When you think about what is happening, the  compute efficiency gains you get from research  

are so large that you actually want most of your  compute to go to research, not to development. 

All these researchers are generating new  ideas, trying them out, testing them,  

and continuing to push the Pareto optimal  curve of scaling laws further and further. 

Empirically, what we’ve seen is that  model costs get ten times cheaper  

every year, or even more than that. At the same scale it gets ten times cheaper,  

and to reach new frontiers it  costs the same amount or more. 

So you don't want to allocate too  many resources to pre-training and RL. 

You actually want to allocate most  of your resources to research. 

In the middle is this development period. If you pre-train a five-trillion-parameter model,  

how many rollouts do you have to do in RL? Rollouts for a five-trillion-parameter model  

are five times larger than for  a one-trillion-parameter model. 

If you wanted to do as many rollouts—maybe  the larger model is two times more sample  

efficient—now you need 2.5x as much  time of RL to get the model smarter. 

Or you could RL the smaller model for 2x the time. You'd still have a 25% difference in the big  

model, which is 2x as sample efficient  and doing X number of rollouts. 

But the smaller model, which is a  trillion parameters, although its  

less sample efficient, is doing twice as  many rollouts and is still done faster. 

You get the model sooner, you've done more RL,  and then you can take that model to help you  

build the next models, help your engineers  train, and do all these research ideas. 

This feedback loop is actually weighed  towards smaller models in every case,  

no matter what your hardware is. As you look to Google, they do  

deploy the largest production model of  any of the major labs with Gemini Pro. 

It's a larger model than GPT-5.4. It's a larger model than Opus. 

Google does this because they have a unipolar set  of compute. It's almost all TPU. Whereas Anthropic  

is dealing with H100s, H200s, Blackwell,  Trainiums, and TPUs of various generations. 

OpenAI is dealing with mostly Nvidia right now,  but going towards having AMD and Trainium as well. 

The fleets of compute like Google's can  just optimize around a larger model. 

They can leverage a thousand chips in a scale-up  domain to get the RL time speed much faster  

so that this feedback loop can be fast. But at the end of the day, in isolation,  

you almost always want to go with a smaller  model that gets RL'd faster and gets deployed  

into research and development earlier. You can build the next thing and  

get more efficiency wins. You have this compounding  

effect of making a smaller model that can be  deployed into research and development earlier. 

I spend less compute on the training because I  was able to allocate more compute to the research. 

This compounding effect of being able  to do research faster and faster is  

potentially a faster takeoff. That's all these companies want:  

the fastest takeoff possible. Okay, a spicy question. You've explained  

that SemiAnalysis sells these spreadsheets. You're always pointing out how six  

months or a year ago, you warned  people about the memory crunch. 

Now you're telling people about the cleanroom  crunch, and in the future, the tool crunch. 

Why is Leopold the only person using your  spreadsheets to make outrageous money? What  

is everybody else doing? I think there are a lot  

of people making money in many ways. Leopold jokes that he's the only client  

of mine who tells me our numbers are too low. Everyone else tells me our numbers are too high,  

almost ad nauseam. Whether it's a hyperscaler saying,  

"Hey, that other hyperscaler, their numbers are  too high," and we're like, "Nah, that's it." 

They're like, "No, no, no, it's  impossible," blah, blah, blah. 

You finally have to convince them through all  these facts and data when we're working with  

hyperscalers or AI labs that in fact, no,  that number isn't too high, that's correct. 

Eventually, sometimes it takes them  six months to realize, or a year later. 

Other clients, on the trading  side, also use our data. 

Roughly 60% of my business is industry. So AI labs, data center companies,  

hyperscalers, semiconductor companies, the  whole supply chain across AI infrastructure. 

But 40% of our revenue is hedge funds. I'm not going to comment on who our customers are,  

but a lot of people use the data. It's just how do you interpret it,  

and then what do you view as beyond it? I will say Leopold is pretty much the only person  

who tells me my numbers are too low, always. Sometimes he's too high, sometimes I'm too low. 

But in general, I think  other people are doing that. 

You can look across the space at hedge funds and  look at their 13Fs and see they own, maybe not  

exactly what Leopold does, because it's always a  question of what is the most constrained thing. 

What's the thing that's going to  be most outside of expectations? 

That's what you're really trying to  exploit: inefficiencies in the market. 

In a sense, our data is making the market  more efficient by making the base data  

of what's happening more accurate. Many funds do trade on information  

that is out there… I don't  think Leopold's the only person. 

I think he has the most conviction  about the AGI takeoff, though. 

Right, but the bets are not  about what happens in 2035. 

The bets that you're making—that are at least  exemplified by public returns we can see for  

different funds including Leopold's—are  about what has happened in the last year. 

The last year stuff could be  predicted using your spreadsheets. 

It's about buying the next year's spreadsheets. They're not just spreadsheets. There are  

reports. There's API access to  the data. There's a lot of data. 

But do you see what I mean? It's not about some crazy singularity thing. 

It's about, do you buy the memory crunch? You only buy the memory crunch if you  

believe AI is going to take off in a huge way. The memory crunch, a lot of it was predicated  

on… At least for people in the Bay Area who  think about infrastructure, it's obvious. 

KV cache explodes as context lengths get longer,  so you need more memory. Then you do the math.  

You also have to have a lot of supply chain  understanding of what fabs are being built,  

what data centers are being built,  how many chips, and all these things. 

We track all these different datasets  very tightly, but at the end of the day,  

it takes someone to fully believe  that this is going to happen. 

A year ago, if you told someone memory  prices would quadruple and smartphone  

volumes are going to go down 40% over the  year or two after that, people were like,  

"You're crazy. That'd never happen." Except a few  people do believe that, and those people did trade  

memory. And people did. I don't think Leopold  was the only person buying memory companies. 

He, of course, sized and positioned and did  things in better ways than some, maybe most. 

I don't want to comment on whose returns  are what, but he certainly did well. 

Other people also did really well. Wow, you've made me diplomatic for  

the first time ever. No, no, you're fine.  I think this is hilarious. I'm being a  

diplomat, whereas usually I'm spicy. Okay, some rapidfire to close out. 

If you're saying with the memory,  logic, et cetera, the N3 is mostly  

going to be AI accelerators, but then there's  N2, which is mostly Apple now… In the future,  

I guess AI would also want to go on N2. Can TSMC kick out Apple if Nvidia and  

Amazon and Google say, "Hey, we're willing  to pay a lot of money for N2 capacity?" 

I think the challenge with this is chip design  timelines take a long while, so that's more  

than a year out, and the designs that are  on two nanometer are more than a year out. 

What would really happen is Nvidia and  all these others will be like, "Hey,  

we're going to prepay for the capacity  and you're going to expand it for us." 

Maybe TSMC takes a little  bit of margin, but not a ton. 

They're not going to kick Apple out entirely. What they're going to do is when Apple orders X,  

they might say, "Hey, we project you only need  X minus one, and so that's what we're going to  

give you, X minus one." Then that flex capacity,  

Apple's kind of screwed on. Traditionally, Apple has always  

over-ordered by 10% and cut back  by 10% over the course of the year. 

Some years they hit the entire 10%. Volumes vary based on the season and macro. 

I don't think TSMC would kick out Apple. I think Apple will become a smaller and smaller  

percentage of TSMC's revenue, and therefore be  less relevant for TSMC to cater to their demands. 

TSMC could eventually start saying, "Hey, you've  got to pre-book your capacity for next year,  

for two years out, and you have to prepay for  the CapEx," because that's what Nvidia and  

Amazon and Google are doing. I wonder if it's worth  

going into specific numbers. I don't have any of them on hand. 

What percentage of N2 does Apple have its  hands on over the coming years versus AI? 

This year Apple has the majority of  N2 that's going to get fabricated. 

There's a little bit from AMD. They are trying to make some AI  

chips and CPU chips early. There's a little bit,  

but for the most part, it's Apple. As we go forward to the year after that, Apple  

still gets closer to half of it as other people  start ramping, but then it falls drastically,  

just like for N3, where they were half. When I say N2, that includes A16,  

which is a variant of N2. Over time, those nodes will be the majority. 

What's also interesting is traditionally,  Apple has been the first to a process node.  

2 nm is actually the first time they're  not. Well, that’s besides Huawei. Huawei,  

back in 2020 and before, was the first with  Apple, but they were both making smartphones. 

Now, with 2 nm, you've got AMD trying  to make a CPU and a GPU chiplet that  

they use advanced packaging to package  together, in the same timeframe as Apple. 

This is a big risk for AMD that causes  potential delays because it's a brand-new  

process technology. It's hard. But at the end  of the day, this is a bet that they want to do  

to scale faster than Nvidia and try and beat them. As we move forward, when we move to the A16 node,  

the first customer there is not even  Apple. It's AI. As we move forward,  

that will become more and more prevalent. Not only will Apple not be the first to a node,  

they will also not be the majority  of the volume to the new node. 

They'll then just be like any old customer. Because the scale of TSMC's CapEx keeps  

ballooning, but Apple's business  is not growing at the same pace,  

they become a less and less relevant customer. They also will just cut their orders because  

things in the supply chain are  kicking them out, whether it be  

packaging or materials or DRAM or NAND. These things are increasing in cost. 

They can't pass on all the cost to customers  likely because the consumer is not that strong. 

You end up with this conundrum  where they are just not TSMC's  

best bud like they have been historically. Do you think if Huawei had access to 3 nm,  

they would have a better accelerator than Rubin? Potentially, yeah. Huawei was the  

first with a 7 nm AI chip as well. They were the first with a 5 nm mobile chip,  

but they were the first with a 7 nm AI chip. The Huawei Ascend was two months before the TPU  

and four months before Nvidia's A100, I think. That's just moving to a process node. 

That doesn't imply software or hardware  design or all these other things. 

But Huawei is arguably the only company in the  world that has all the legs. Huawei has cracked  

software engineers. Huawei has cracked  networking technologies. That's, in fact,  

their biggest business historically. They have  cracked AI talent. Furthermore, beyond Nvidia,  

they actually have better AI researchers. Beyond Nvidia, they have their own fabs. 

And beyond Nvidia, they have their own end  market of selling tokens and things like that. 

Huawei is able to get the top, top talent. Nvidia is as well, but not with as much  

concentration, and Huawei  has a bigger pool in China. 

It's very arguable that Huawei, if they  had TSMC, would be better than Nvidia. 

There are areas where China has advantages  in areas that Nvidia can't access as easily. 

Not just scale, but certain optical  technologies China's actually really good at. 

I think it's very reasonable that if in  2019 Huawei was not banned from using TSMC,  

Huawei would have already eclipsed  Apple as the biggest TSMC customer. 

Huawei has huge share in networking,  compute, CPUs, and all these things. 

They would have kept gaining share, and  they'd likely be TSMC's biggest customer. 

Wow. That's crazy. I've got a  random final question for you. 

The other part of the Elon interview was robots. If humanoids take off faster than people expect,  

if by 2030 there's millions of humanoids  running around which each need local compute,  

any thoughts on what that implies? What would be required for that? 

There's a lot of difficulties with the VLMs  and VLAs that people are deploying on robots. 

But to some extent, you don't need to  have all the intelligence in the robot. 

It would be much more efficient to not do that. Because in the cloud, you can batch  

process and all these things. What you may want to do is have a  

lot of the planning and longer-horizon tasks  determined by a much more capable model in  

the cloud that runs at very high batch sizes. Then it pushes those directions to the robots,  

who interpolate between each subsequent action. Or it is given a command like, "Hey,  

pick up that cup," and then the model  on the robot can pick up the cup. 

As it's picking up, things like weight and  force may have to be determined by the model  

on the robot, but not everything needs to be. It can say, "hey that’s a headphone" and the  

super model in the cloud can say, "I  know these headphones are Sony XM6s,"  

which is not a Dwarkesh ad spot, but... I’m like, why is this guy's plugging this  

thing so hard. It's on the table. It's on his  neck when we're interviewing Satya together. 

Is he getting paid by Sony? Unfortunately not. But anyways,  

it might say, "Hey, the headband is soft, and  this is the weight of it," and all these things. 

Then the model on the robot  can be less intelligent,  

take these inputs, and do the actions. It may get told by the model in the cloud  

every second, or maybe ten times a second,  depending on the hertz of the action. 

But a lot of that can be offloaded to the cloud. Otherwise, if you do all of the processing on the  

device, I believe it would be more  expensive because you can't batch. 

Two, you couldn't have as much intelligence  as you do in the cloud because the  

models will just be bigger in the cloud. Three, we're in a semiconductor shortage world,  

and any robot you deploy needs leading-edge  chips because the power is really bad for robots. 

You need it to be low power and efficient,  and all of a sudden you're taking power  

and chips that would've been for AI data  centers, and you're putting them in robots. 

So now that 200 gigawatts gets lower if  you're deploying millions of humanoids. 

I think this is very interesting because  something people might not appreciate  

about the future is how centralized, in  a physical sense, intelligence will be. 

Right now, there are eight billion humans, and  their compute is in their heads, on their person. 

In the future, even with robots that are  out physically in the world—obviously,  

knowledge work will be done in a centralized  way from data centers with hundreds of thousands  

or maybe millions of instances—the future  you're suggesting is one where there's more  

centralized thinking and centralized computation  driving millions of robots out in the world. 

That's an interesting fact about the future  that I think people might not appreciate. 

I think Elon recognizes this, which is why  he's going to different places for his chips. 

He signed this massive deal with Samsung to make  his robot chips in Texas because I personally  

think he thinks Taiwan risk is huge. Because of that and the centralization  

of resources in Taiwan, having his robot  chips in Texas means having a separate  

supply chain that is not as constrained. No one's really making AI chips on Samsung  

besides Nvidia's new LPU that they launched. They’re launching it next week, but we're  

recording this the week before. This episode's coming out Friday. 

Oh, this episode's coming out before.  Sick. They're launching this new AI chip  

next week which is built on Samsung, but  that's a recent development from Nvidia. 

That's the only other AI demand there,  whereas on TSMC, everything is competing. 

He gets both geopolitical diversification  and supply chain diversity for his robots,  

and he's not competing as much with the infinite  willingness to pay for the data center geniuses. 

Final question, on Taiwan. If we believe  that tools are the ultimate bottleneck,  

how much of Taiwan's place in the AI semiconductor  supply chain could we de-risk simply by having a  

plan to airlift every single process engineer  at TSMC out if they get blockaded or something? 

Or do you still need to ship out the EUV  tools, which would be multiple plane loads  

per single tool and would not be practical? If you ship out all the process engineers and  

assuming it's hot enough that you destroy the  fabs, no one has all the fabs in Taiwan now,  

which is a big risk. These tools actually use a lot of  

semiconductors which are manufactured in Taiwan. It's a snake eating its own tail meme because  

you can't make the tools without the chips from  Taiwan, which you can't use without the tools in  

Taiwan. There's obviously some diversification  there. They don't use super advanced chips in  

lithography tools, but at the end of the  day, there is some dragon eating its tail. 

Just shipping out all the engineers and  blowing up the fabs means China has a  

stronger semiconductor supply chain than the  rest of the world in terms of verticalization,  

now that you've removed Taiwan. You've got all the know-how,  

but you've got to replicate it in,  let's say, Arizona or wherever for TSMC. 

It's going to take a long time to build all the  capacity that TSMC has built over the years. 

And so you've drastically  slowed US and global GDP. 

Not just growth, you've shrunk the GDP  massively, and you've got a lot bigger problems. 

Your incremental ability to add  compute goes to almost zero. 

Instead of hundreds of gigawatts  a year by the end of the decade,  

let's say something happens to Taiwan, now you're  at maybe 10 gigawatts across Intel and Samsung,  

or 20 gigawatts. It's nothing. Now all of a sudden  you've really caused some crazy dynamics in AI. 

Of course, you have all the existing capacity,  but that existing capacity pales in comparison  

to the capacity that's being expanded. Okay. Dylan, that was excellent. Thank  

you so much for coming on the podcast. Thank you for having me. And see you tonight.