super dense and high quality discussion of AI hardware bottlenecks

Anthropic and openai will have about the same amount of compute at EoY despite openai raising way more money (OAI just has more stuff in the pipeline long term I think/anthropic does more renting, less owning?)

An H100 is worth more today than it was three years ago, bc the value per output token is higher

These are appreciating assets it turns out??

About 3.5 EUV machines are needed per gigawatt per year, and asml makes about 70-80 machines per year rn, could plausibly ramp up to 100 in 5 years

Memory is also a big bottleneck right now, uses the same leading edge process

Using commoditized slightly worse memory isn't really an option because the bandwidth would be bad enough that you wouldn't be able to keep the compute core busy

Consumer electronics are likely to increase significantly in price because of the scarce memory because of scarce fabs (even more than they already have)

One way around the electrician bottleneck is just to assemble more of it in a factory before it comes to the site

Space datacenters don't make sense

Everything is harder and slower

And Dylan isn't that worried about the permitting bottleneck