Elon Musk reveals how many Nvidia H100 chips his AI chatbot will be trained on
Elon Musk said Grok 3 will be trained on 100,000 Nvidia H100 GPUs.
Elon Musk is hyping up upcoming versions of his AI chatbot, Grok.
The billionaire replied to a post on X on Monday and said that the latest version of xAI’s chatbot Grok 3 should be “something special’ after it trains on 100,000 H100s.
Musk is referring to Nvidia’s H100 graphics processing unit, also known as Hopper, which is an AI hip that helps handle data processing for large language models (LLMs). The chips are a key component of AI development and a hot commodity in Silicon Valley as tech companies race to build ever-smarter AI products.
Each Nvidia H100 GPU chip is estimated to cost around $30,000, although some estimates place the cost as high as $40,000. Volume discounts may also be possible.
Based on those estimates, that would mean Grok 3 is being trained on $3 billion to $4 billion worth of AI chips — but it’s not clear if those chips were purchased outright by Musk’s company. It’s also possible to rent GPU compute from cloud service providers, and The Information reported in May that Musk’s xAI startup was in talks with Oracle to spend $10 billion over multiple years to rent cloud servers.
But we do know that Musk’s companies have purchased a hefty amount H100s outright in recent years. The Tesla CEO reportedly diverted a $500 million shipment of Nvidia H100s intended for Tesla to X instead, for example.
Training based on 100,000 GPUs would be a big step up from Grok 2. Musk said in an interview in April with the head of Norway’s sovereign fund Nicolai Tangen that Grok 2 would take around 20,000 H100s to train.
xAI has so far released Grok-1 and Grok-1.5, with the latest only available to early testers and existing users on X, formerly known as Twitter. Musk said in a post on X Monday that Grok 2 is set to launch in August and indicated in the other post about GPUs that Grok 3 will come out at the end of the year.
xAI did not respond to a request for comment.
100,000 GPUs sounds like a lot — and it is. But other tech giants like Meta are stacking up on even more GPUs. Mark Zuckerberg said in January that Meta will have purchased about 350,000 Nvidia H100 GPUs by the end of 2024. He also said Meta will own about 600,000 chips including other GPUs.
If that’s the case, Meta will have spent about $18 billion building its AI capabilities.
The stockpiling of H100 chips has also contributed to how ruthless hiring top AI talent has become in the last year.
Aravind Srinivas, founder and CEO of AI startup Perplexity, talked about getting turned down by a Meta AI researcher he was trying to poach in part because of Zuckerberg’s huge collection of AI chips.
“I tried to hire a very senior researcher from Meta, and you know what they said? ‘Come back to me when you have 10,000 H100 GPUs,'” Srinivas said.
