Why Do AI Costs Keep Rising Across Models?

TL;DR

AI costs are rising because output tokens and reasoning overhead can far exceed input token costs, even as per-token input prices fall. The biggest drivers are how much the model generates in tokens and the inclusion of reasoning steps, which inflate overall cost, making some cheaper input models dramatically more expensive to run in practice.

Transcript

A while back, I published a video called AI a race to the bottom. It was about how the price of the different models keeps getting cheaper and how exciting that was to me about what we could do with these things. And on one hand, I was kind of right, but on the other, I was really, really wrong. It actually feel like I was a little bit misled. When... Read More

Key Insights

• Tokens are the core price units, with input costs and output costs priced separately, shaping overall expense.
• Reasoning increases output token generation, which can massively inflate costs even when input costs are cheap.
• Some models appear cheap to input but generate many tokens in output, driving up total cost.
• Output token cost, not just input cost, often dominates the final bill in benchmarks.
• Model pricing instability and differing token handling create misleading cost comparisons when not all factors are shown.
• Benchmarks show that a cheaper input model can be far more expensive due to long reasoning and high output tokens.
• The cost contrast between models like Grok 4 and Claude for Opus illustrates how reasoning tokens skew totals.
• Techniques and tools that simplify usage and unify APIs can help reduce practical cost by lowering unnecessary token generation.
• More videos from this channel:
  • What is GPT6.1 Soul and why it costs less
  • How to build fast lightweight websites with HTML
  • How good is GPT-5.2 really and where does it fail
• More from this channel: Theo - t3․gg

Questions & Answers

Q: A real search query question about the video (e.g. 'How to...', 'What is...', 'Why does...', 'When should...'). Question 1

Why do AI costs sometimes rise even when token input prices fall, and how does output token generation contribute to this trend? The video explains that while input costs may decrease, many models incur high expenses due to the number of tokens produced as output and the overhead from model reasoning. This means that a model with cheap input tokens can be much more expensive in practice if it produces long outputs or performs extensive reasoning, which increases total cost substantially.

Q: Question 2

What role does reasoning play in the cost of running AI benchmarks across models, and how does it differ between non-reasoning and reasoning-enabled models? Reasoning causes models to generate far more output tokens, which can drastically increase price. A non-reasoning model might output a short result, whereas a reasoning-enabled model can produce many tokens to explain its steps, leading to much higher costs for the same input.

Q: Question 3

How does Grok 4 compare to Claude for Opus in terms of cost and performance, according to the transcript? The transcript notes Grok 4 is more expensive than Claude for Opus when run on the same benchmark, despite cheaper input costs in some charts. The discrepancy is attributed to Grok 4 producing many more tokens due to its reasoning style, inflating its total cost.

Q: Question 4

Why is the cost analysis of AI models sensitive to whether you include artificial analysis or reasoning in the benchmark? Because including reasoning tokens can dramatically inflate output costs, making models that reason more appear much more expensive, even if input costs are low. The transcript emphasizes that total cost depends heavily on how many tokens are generated during reasoning.

Q: Question 5

What does the video suggest about comparing model costs using only input token prices? It suggests that such comparisons can be misleading because they ignore the large influence of output tokens and reasoning overhead. A full cost analysis should include both input and output costs and consider the model's tendency to generate tokens.

Q: Question 6

How do token costs and model pricing interact to shape total expense in AI workloads, based on the transcript? Token costs are the base, but the total expense is dominated by output tokens and reasoning, which can dwarf input costs. The dynamic means pricing charts must reflect both sides to be meaningful for developers planning workloads.

Q: Question 7

What practical advice does the video give for developers evaluating AI models, given rising costs? The practical advice is to benchmark with realistic tasks, compare both input and output token costs, and be wary of models that generate lengthy reasoning if cost is a concern. Prefer configurations that minimize unnecessary token generation while meeting task requirements.

Q: Question 8

What is the broader implication of the described pricing dynamics for AI tool adoption and subscriptions, as hinted by the description and sponsor mention? The broader implication is that pricing models and subscriptions may get squeezed as token volume increases with model capabilities. Developers should consider total cost of ownership, including potential bursts in output tokens, when selecting tools and evaluating ROI.

Summary & Key Takeaways

• The video argues that while per-token input costs have fallen, total cost is driven by how many tokens are produced during output and by the model’s reasoning, which can dramatically inflate the bill. It compares different models and highlights cases where cheaper input tokens still lead to higher totals due to long outputs and reasoning. The takeaway is that cost analysis must consider both input and output token economics and the impact of reasoning on price.

• The author shares personal experience benchmarking multiple AI models, noting Grok 4 and other high-output models can be far more expensive even when input costs look low. The comparison demonstrates that price charts can be misleading if they only show input costs or overlook output token dynamics. Practical advice is to test models with realistic tasks.

• The description also advertises a sponsor and references an external article about subscriptions and price squeezing, implying that the economics of AI pricing are complex and evolving, with real-world implications for developers and users who depend on predictable costs and scalable deployment.