The chipmaker, now the most valuable public company in the world, said strong demand for its chips should continue this quarter.
The chipmaker, now the most valuable public company in the world, said strong demand for its chips should continue this quarter.
Nods vigorously.
The future of LLMs basically unprofitable for the actual AI companies. We are in a hell of a bubble, which I can’t wait to pop so I can pick up a liquidation GPU (or at least rent one for cheap).
That doesn’t mean power usage is an existential issue. In fact, it seems like the sheer inefficiency of OpenAI/Grok and such are nails in their coffins.
Power usage is what’s sucking the cash. What else could it be? Not all of these companies are building out lots of datacenters the way OpenAI is. They built what they have, and are now trying to make money on it.
The companies that are charging for AI are charging about as much as buyers are willing to pay, but it’s orders of magnitude too small to cover their costs. The big cost is power usage.
On the training side, it’s mostly:
Paying devs to prepare the training runs with data, software architecture, frameworks, smaller scale experiments, things like that.
Paying other devs to get the training to scale across 800+ nodes.
Building the data centers, where the construction and GPU hardware costs kind of dwarf power usage in the short term.
On the inference side:
Sometimes optimized deployment frameworks like Deepseek uses, though many seem to use something off the shelf like sglang
Renting or deploying GPU servers individually. They don’t need to be networked at scale like for training, with the highest end I’ve heard (Deepseek’s optimized framework) being like 18 servers or so. And again, the sticker price of the GPUs is the big cost here.
Developing tool use frameworks.
On both sides, the biggest players burn billions on Tech Bro “superstar” developers that, frankly, seem to Tweet more than developing interesting things.
Microsoft talks up nuclear power and such just because they want to cut out the middleman from the grid, reduce power costs, reduce the risk of power outages and such, not because there’s physically not enough power from the grid. It’s just corporate cheapness, not an existential need.