Jonathan Ross on AI Compute Demand and Nvidia's Future

TL;DR

Demand for AI compute is insatiable, according to Groq founder Jonathan Ross, who believes OpenAI and Anthropic would nearly double revenue within a month if their inference compute doubled. He argues hyperscalers keep spending because they fear being locked out of their business, and he would be surprised if Nvidia were not worth 10 trillion dollars in 5 years.

Transcript

The countries that control compute will control AI. And you cannot have compute without energy. So, I'm thrilled to welcome Jonathan Ross, founder and CEO Grock, back to the hot seat. And now we're going to be able to add more labor to the economy by producing more compute and better AI. That has never happened in the history of the economy before.... Read More

Key Insights

• The better question than whether AI is a bubble is what the smart money is doing. Google, Microsoft, Amazon and some nations are all doubling down on AI, and each spending announcement tends to be larger than the last one.
• Microsoft showed the value of AI compute when, in one quarter, it deployed GPUs and chose not to offer them in Azure, because using them internally made more money than renting them out to customers.
• The AI market resembles the early days of oil drilling: many dry holes and a few gushers. Around 35 or 36 companies reportedly account for 99% of AI revenue or token spend, which shows how lumpy the market is.
• Hyperscaler spending is not driven by a purely economic framework. Ross argues it is about maintaining leadership, because the alternative is being completely locked out of their business, so they spend heavily on AI.
• Vibe coding already delivers real ROI. At Groq, a customer-requested feature went to production in 4 hours with no human-written code and no human debugging, done entirely through prompting and even committed through Slack.
• Building AI chips is extremely hard. Google ran about three chip efforts at once and only one, the TPU, outperformed GPUs, and other projects such as Dojo were recently cancelled.
• Nvidia investing in OpenAI is not an infinite money loop, according to Ross, because at least 40% of the spend on infrastructure goes out into the wider ecosystem to suppliers rather than back to Nvidia.
• Speed matters commercially. Ross compares AI to consumer packaged goods, where high margins correlate with how quickly an ingredient acts on you, because a fast dopamine cycle builds brand affinity.

Questions & Answers

Q: Is AI a bubble according to Groq founder Jonathan Ross?

Ross suggests reframing the question. Instead of asking whether AI is a bubble, ask what the smart money is doing. Google, Microsoft, Amazon and some nations are all doubling down, and each spending announcement grows. He believes that, in aggregate, less money will go into AI than comes out, even though many individual investors will lose money because the market is very lumpy.

Q: Why do hyperscalers keep increasing AI capex spending?

Ross says hyperscalers are not using a purely economic framework. At a Goldman Sachs event in Abu Dhabi, he asked investors managing 10 billion dollars or more whether they were 100% convinced AI could not do their job in 10 years, and no hands went up. Hyperscalers feel the same, so they spend heavily because the alternative is being locked out of their business.

Q: Why would OpenAI and Anthropic revenue double with more compute?

Ross argues both companies are compute limited. One of the biggest complaints about Anthropic is rate limits, since people cannot get enough tokens; more compute means more tokens and more revenue. OpenAI runs a chat service, and the way to regulate it with limited compute is to run it slower, which reduces engagement. He wagers doubling inference compute would almost double their revenue within a month.

Q: Is Nvidia investing in OpenAI an infinite money loop?

Ross says it would be round-tripping only if no productive outcomes occurred. He estimates at least 40% of the spend goes to building infrastructure through suppliers, so those dollars flow into the wider ecosystem. Around 60% may return to Nvidia, and the stock gains reflect a belief that revenue will continue, which he thinks is likely true because there is not enough compute in the world.

Q: Should big tech companies build their own AI chips?

Ross doubts many will succeed. He notes Google had about three chip efforts running at the same time and only one, the TPU, outperformed GPUs, while efforts like Dojo were recently cancelled. He compares building a chip to compete with Nvidia to replicating Google search. Still, if several players try and one succeeds, the company gains optionality through another chip.

Q: How is vibe coding delivering ROI for companies today?

Ross describes a customer meeting at Groq where the customer asked for a feature. He specified it at a high level, prompting the engineers, and 4 hours later it was in production. No human wrote code and no human debugged it. He expects that within 6 months this could happen before the meeting ends, a qualitative change that helps win deals competitors cannot.

Q: Why is the AI market compared to early oil drilling?

Ross says the AI market is like early oil drilling: many dry holes and a few gushers. People invest on instinct, almost vibe investing, and those with good instincts make fortunes while others lose money. About 35 or 36 companies reportedly drive 99% of AI revenue or token spend. Over time it will become a predictable science, and good investors will make less money.

Q: Why does inference speed matter for AI companies?

Ross uses consumer packaged goods as an analogy. Ranking CPG products by margin puts smoking tobacco at the top, then chewing tobacco, soft drinks and eventually water. The factor that correlates with high margin is how quickly the ingredient acts on you. A fast dopamine cycle creates brand affinity, suggesting faster AI responses build stronger engagement and loyalty.

Summary & Key Takeaways

• Jonathan Ross, founder and CEO of Groq and previously the leader of the team that built Google's first TPU, argues that instead of asking whether AI is a bubble, people should watch the smart money. Google, Microsoft, Amazon and nations are all spending more on AI with every announcement.

• Ross compares the AI market to early oil drilling, with many dry holes and a few gushers. He says hyperscalers spend heavily because they fear being locked out of their business, and that AI already returns large value, citing a feature Groq shipped to production in 4 hours with no human-written code.

• Ross believes compute demand is insatiable and that OpenAI and Anthropic would almost double revenue within a month with double the inference compute. He doubts many companies will succeed at building chips, says Nvidia's OpenAI investment is not a pure loop, and would be surprised if Nvidia were not worth 10 trillion dollars in 5 years.