How Will Cheaper AI Tokens Reshape Software?

TL;DR

Falling AI token prices could pressure expensive model providers and software companies that pass high inference costs to customers. Chamath Palihapitiya argues that most use cases can rely on models that are 80–95% as capable, while premium models remain worthwhile for narrow applications that generate substantial incremental revenue.

Transcript

GO CRY IN YOUR BEER. ISN'T THAT YOUR BEER CRYING YOUR COFFEE. AND AND WE'RE GOING TO GET TO TO CHAMATH. THANKS, STEPH. >> OKAY. OKAY. THANKS TO BRING IN OUR SPECIAL GUEST THIS MORNING. HE IS SITTING ON SET WATCHING ALL OF THIS PLAY OUT. HASN'T BEEN HERE IN A BIT AND WE'RE THRILLED TO HAVE HIM BACK. JAMAL IS HERE. HE'S THE FOUNDER AND CEO OF SOCIAL ... Read More

Key Insights

• AI profitability is still uncertain because model providers, downstream software companies, and token buyers all need viable economics. Rapid revenue growth at a few companies does not establish where profits will emerge across the broader ecosystem or whether customers will earn adequate returns.
• Token prices vary dramatically across providers, according to Palihapitiya's comparison. He describes a million tokens as a barrel of intelligence costing $26 from some providers, $56 for Anthropic's latest model, about $1 from Grok, $1.50 from Meta, and $0.50 from Chinese providers.
• Uncontrolled token consumption could become a material operating expense inside large organizations. Palihapitiya predicts that some chief executives and finance leaders may discover the scale of internal usage only after earnings fall short, prompting them to compare costly tokens with much cheaper alternatives.
• AI hardware and memory could remain commercially strong for another couple of years because their supply ecosystem faces significant constraints. Palihapitiya distinguishes this scarcity-driven opportunity from uncertainty surrounding software pricing, model valuations, and the ability of downstream customers to monetize token consumption.
• Large language models are converging in capability rather than producing the dramatic performance leaps seen in earlier releases. As models increasingly support similar everyday behaviors, customers have a rational reason to question premium prices and select cheaper systems for routine workloads.
• Cheaper models are sufficient for most use cases, according to Palihapitiya, while a limited group of specialized applications still justifies premium pricing. A cybersecurity company can rationally buy expensive intelligence when better infrastructure protection creates billions of dollars in additional revenue.
• Leading AI labs face power, data-center, and usage constraints that can limit access even for paying customers. Anthropic's reconsideration of reduced model access illustrates the competitive risk that subscribers may cancel and move to OpenAI when restrictions or additional charges become unacceptable.
• Google and Meta are gaining competitive footing by repeatedly releasing models that Palihapitiya estimates are 80–95% as good as leading systems. Their available capacity and lower pricing create a difficult market dynamic for labs that depend on selling substantially more expensive model access.

Questions & Answers

Q: Why could falling AI token prices hurt software companies?

Falling token prices expose the gap between companies that use expensive model access and competitors that obtain similar capabilities more cheaply. Software businesses that pass high token costs through to customers may encounter resistance, especially when less expensive models handle most tasks adequately. The resulting pressure could reduce margins, force pricing changes, and make existing commitments to costly providers harder to justify.

Q: What does a barrel of intelligence mean in AI economics?

A barrel of intelligence is Palihapitiya's analogy for one million AI tokens. He compares it with a barrel of oil to show how providers sell a broadly similar input at sharply different prices. His examples range from $56 for Anthropic's latest model to roughly $1 from Grok and $0.50 from Chinese providers, illustrating the potential for price rationalization.

Q: How can token usage cause companies to miss earnings estimates?

Employees and systems may consume large quantities of tokens without the chief executive or chief financial officer fully understanding the accumulated cost. Palihapitiya suggests that this hidden spending could eventually appear as unexpected operating expense, causing earnings per share to miss expectations by a few pennies. Management may then trace the shortfall to expensive model usage occurring throughout the organization.

Q: Are cheaper AI models good enough for business use?

Cheaper AI models are good enough for most use cases, according to Palihapitiya, because competing systems have increasingly converged in capability. He says models from capacity-rich providers are reaching 80–95% of the quality of leading alternatives. The remaining gap matters in some specialized applications, but routine behavior may not justify paying many times more for every token.

Q: When should a company pay for an expensive AI model?

A company should pay for an expensive model when a narrow, high-value application produces enough additional revenue or protection to justify the cost. Palihapitiya uses cybersecurity as an example, arguing that a company such as Palo Alto Networks could rationally buy premium intelligence if it helps secure major corporate infrastructure and supports billions of dollars in incremental revenue.

Q: Why are leading AI labs constrained despite strong demand?

Leading labs are constrained by limited power, data-center capacity, and available usage. These restrictions make it difficult to provide unrestricted access even to paying customers. The Anthropic example shows the commercial tension: reducing included access or requiring a more expensive plan may control consumption, but customers can respond by canceling and switching to another provider with an excellent model.

Q: How are Google and Meta changing AI model competition?

Google and Meta are changing competition by using their capacity to release model after model at lower prices while approaching the quality of leading systems. Palihapitiya estimates that these releases are 80–95% as good. That combination of adequate performance, repeated improvements, and greater infrastructure capacity creates a complicated competitive position for OpenAI, Anthropic, and other premium-priced laboratories.

Q: Why could AI hardware outperform AI software investments?

AI hardware and memory benefit from significant scarcity and infrastructure constraints, which Palihapitiya expects to persist for another couple of years. Software economics are less settled because providers and customers still must determine how token spending creates profit or growth. Consequently, chips, memory, and related hardware can remain commercially strong even while expensive models and downstream software face pricing pressure.

Summary & Key Takeaways

• AI economics depends on whether downstream software companies and their customers can turn token spending into higher profits or faster growth. Palihapitiya warns that organizations may be consuming expensive tokens without executives fully understanding the resulting operating expenses, creating a risk of future earnings misses linked to uncontrolled AI usage.

• Model capabilities are converging, making price increasingly important for common tasks. Palihapitiya compares successive releases to newer iPhones that support largely similar behaviors. He argues that inexpensive models are sufficient for most applications, while costly premium intelligence remains rational for specialized uses that can directly produce billions of dollars in incremental revenue.

• Hardware, memory, power, and data-center capacity remain constrained, supporting continued strength in that part of the AI market. Meanwhile, Google and Meta are using their capacity to release models that Palihapitiya describes as 80–95% as good as leading alternatives, complicating the position of labs offering more expensive models.