"A Billion Times" A.I. Inference Increase
Nvidia surges, again; Google buys 500 MW of nuclear power; Oklo gets DOE approval; Amazon invests in X-energy.
Building A.I. models – or “training” them with exabytes of historical data – gets most of the attention. But over time “inference” – or asking the trained models to generate answers – will dominate A.I. workloads.
Nvidia CEO Jensen Huang just made this point in an interview with venture capitalist Brad Gerstner. Today, around 40% of Nvidia’s business is for A.I. training, and another 40% is for A.I. inference. (The remaining 20% of revenue comes from its traditional video, gaming, and automobile products.)
Inference, however, “is about to go up by a billion times,” Huang says. “That’s the part that most people haven’t completely internalized…This is the Industrial Revolution…It’s going to go up a billion times.”
Nvidia’s Big Tech A.I. customers agree, and they are scrambling for both the chips and electricity to make it happen.
On Tuesday, Google inked a deal with Kairos Power, a startup supplier of small modular reactors (SMR), to purchase up to 500 megawatts of nuclear power by 2035. Meanwhile, Oklo, which is backed by OpenAI, received a design approval this week from the Department of Energy for its Idaho National Laboratory plant, scheduled to go live in 2027. And Amazon, which several months ago bought a data center situated at the Susquehanna, Pennsylvania, nuclear plant, just announced a separate $500 million investment in SMR developer X-energy. The biggest move was of course Microsoft’s $15-billion long-term deal to restart Three Mile Island’s 835-megawatt Unit One reactor.
You will notice, however, that the vast majority of capacity from the welcome and long-overdue revival of nuclear power won't occur in the near term. If they start now, and everything goes according to plan, the nuclear innovators could begin delivering new power later this decade and much more in the 2030s.
A.I. data center builders aren’t so patient. Elon Musk, to Jensen Huang’s total amazement, just locked, loaded, and connected 100,000 H100 chips in his new Memphis, Tennessee, complex in 19 days. This process, Huang says, might commonly take battalions of normal humans one year. The A.I. race is ON!
Nvidia’s new Blackwell generation of chips, moreover, are so power-hungry they mandate a total rethink of data center design. In fact, Meta (Facebook) just demolished and rebuilt a brand new data center because it didn’t conform to the new power-dense and liquid super-cooled design specs.
A new report from Epoch AI suggests that power is the chief constraint to scaling A.I. through 2030.
Another new report from SemiAnalysis projects power needs for all North American data centers will grow to 45,614 MW in 2025 from 24,646 MW in 2023 – a two year rise of 85%. It projects power needs for data centers dedicated to A.I. will grow to 17,883 MW in 2025 from 2,537 MW in 2023 – or around 600%.
Such hyper-growth suggests that despite the exciting but nascent nuclear revival, the A.I. computing boom will, for the next half-decade or more, be powered mostly by traditional and especially by reliable energy sources. That means baseload electricity generated chiefly by natural gas and back-up generators fueled with diesel.
Bret you are over the target! I don’t understand how this isn’t front and center for the “energy warriors” who connect every snowstorm, fire, flood, drought, or afternoon shower with carbon centered climate change. And we are now competing for electrical energy in an increasingly demand hungry future, where supply can only be increased incrementally. I think rooftop solar where appropriate is fantastic, and a good hedge against supply disruptions (blackouts), but with batteries it is pricey. Owning your power production is a good idea if you can afford it, especially if you can store the electrons for when you are going to need them.
It makes you SICK ( actually it probably will) they preach to the likes of me about don't put your home fire on,switch the lights off. You'll kill the planet if you don't put that empty baked bean can in the right bin. Meanwhile THEY,ALL OF THEM are blasting power spillage into the universe like a drunkard on the eve of prohibition. I've read of how much power,that has to be generated first is used to power AI (artificial idiocy) and all these fuckin bombs they're dropping all over the place that must pack the environment with toxins. People can write stories,draw pictures,hoover the floor. I know there is film from way,way back,I never saw it but I read a review of it. It's from the 1970s I think. It stars Julie Christie as a woman who lives in a totally computer regulated home ( how's that for prescient,and the computer rapes her. Let's hope that ones not in development,bet it is.