How Is AI Entering Its Second Scaling Phase?

TL;DR

AI progress now depends on three multiplicative scaling mechanisms: larger pre-training runs, reinforcement learning with verified rewards, and additional compute during inference. Reasoning methods sustained progress while Blackwell deployments lagged, and Gemini 3 indicated that pre-training scaling remained intact. Investors evaluating this transition should test premium models directly and closely follow researchers at the leading laboratories.

Transcript

I will never forget when I first met Gavin Baker. It was early days of the podcast and he was one of the first people I talked to about markets outside of my area of expertise which at the time was quantitative investing about the incredible passionate experience that he's had investing in technology across his career. I find his interest in market... Read More

Key Insights

• • Serious model evaluation requires testing the highest paid tier because free AI tiers may display capabilities resembling a much younger and less developed system. Baker argues that investors cannot responsibly infer a frontier model's full potential from limited free access.
• • AI research signals are concentrated among roughly 500 to 1,000 people who deeply understand the frontier, including a substantial number in China. Following their public discussions, papers, disputes, and podcast appearances can reveal changes before they reach broader investor narratives.
• • Gemini 3 is evidence that pre-training scaling remained intact after a period with few meaningful hardware tests. The scaling relationship is an empirical observation measured with precision, but researchers still lack a complete explanation for why adding resources produces greater intelligence.
• • Reasoning bridged an approximately 18-month infrastructure gap by enabling model progress without waiting for the next generation of Nvidia GPUs or Google TPUs. Without reasoning, Baker believes progress could have stalled from mid-2024 until Gemini 3 became available.
• • Reinforcement learning with verified rewards creates a scaling path after pre-training because outputs can be checked against known criteria. The discussion emphasizes a related principle from Andrej Karpathy: tasks that can be specified and verified are candidates for AI-driven automation.
• • Test-time compute improves results by allowing a model to spend more computation while producing an answer. Together with verified-reward reinforcement learning, it accounted for the immense progress described from October 24 through the time of the conversation.
• • Blackwell deployment is a full-system engineering transition because racks moved from air cooling to liquid cooling, from roughly 1,000 pounds to 3,000 pounds, and from approximately 30 kilowatts to 130 kilowatts. Density also made heat removal unusually difficult.
• • AI scaling mechanisms are multiplicative because post-training and test-time methods can be applied to increasingly capable base models. Gemini 3 therefore matters beyond one product release, since its stronger pre-trained foundation can amplify the benefits of the newer reasoning approaches.

Questions & Answers

Q: How should investors evaluate a newly released AI model?

Investors should begin by using the model themselves and testing its highest paid tier, not merely its free version. Baker compares judging a premium model through a free tier to predicting a 35-year-old's abilities by observing a 10-year-old. Direct testing should be combined with close attention to frontier researchers, new papers, public technical discussions, and podcasts featuring employees from leading laboratories.

Q: Why was Gemini 3 important for AI scaling laws?

Gemini 3 was important because it indicated that the scaling relationship for pre-training remained intact. Baker stresses that this relationship is not a fully understood scientific law, but a precisely measured empirical observation that has held over time. The release provided a meaningful test after newer TPU generations became available, suggesting that larger pre-training efforts could still produce more capable base models.

Q: Why are pre-training scaling laws still poorly understood?

Pre-training scaling laws are poorly understood because researchers can measure their effects without fully explaining their underlying mechanism. Baker compares the situation to ancient societies that precisely tracked the sun and aligned structures with equinoxes while lacking an understanding of orbital mechanics. Similarly, AI researchers observe that additional compute, chips, power, and data produce improved intelligence, but they do not completely know why.

Q: How did reasoning models sustain AI progress during hardware delays?

Reasoning models created progress through reinforcement learning with verified rewards and additional compute at test time. These approaches did not require the next large pre-training run to deliver improvements. Baker argues that reasoning bridged an approximately 18-month period when Hopper clusters had reached practical coherence limits and Blackwell systems were not yet broadly deployed, preventing a prolonged slowdown in visible AI capability.

Q: What makes verified rewards important for AI development?

Verified rewards matter because reinforcement learning can improve a model when the quality of its output can be checked against a reliable result. The transcript connects this idea to Andrej Karpathy's distinction between specification and verification: what can be specified may be automated, and what can be verified may also be automated. Verification therefore defines where post-training can deliver dependable, scalable capability gains.

Q: Why was Nvidia's Blackwell transition difficult?

The Blackwell transition required changes across power, cooling, weight, and physical infrastructure. Compared with Hopper-era systems, the described rack moved from air cooling to liquid cooling, increased from roughly 1,000 pounds to 3,000 pounds, and rose from about 30 kilowatts to 130 kilowatts. The dense design also made heat removal difficult, delaying large-scale deployments until the preceding three or four months.

Q: How do Google TPUs compete with Nvidia GPUs in AI infrastructure?

Google and Nvidia represent competing approaches to AI infrastructure. Google uses its own TPU alongside other networking chips, while Nvidia offers a broader full stack. Google released TPU v6 in 2024 and TPU v7 in 2025, allowing Gemini 3 training to use newer hardware while Blackwell faced a difficult ramp. Baker characterizes this timing difference as a meaningful temporary advantage for Google's training capabilities.

Q: Which sources provide the strongest signals about frontier AI?

The strongest signals come from directly using premium models and following people working near the technical frontier. Baker highlights public activity on X, newly released research papers, exchanges among laboratory teams, Andrej Karpathy's writing, and podcast appearances by employees of OpenAI, Gemini, Anthropic, and xAI. He also uses AI to discuss podcast sections immediately, keeping the research process fast and low friction.

Summary & Key Takeaways

• Gavin Baker evaluates new AI systems by using their highest paid tiers instead of drawing conclusions from free versions. He supplements direct testing by following researchers on X, reading influential technical commentary repeatedly, listening to laboratory employees on podcasts, and discussing important material with AI tools immediately to reduce friction in his research process.

• Gemini 3 mattered because it provided evidence that pre-training scaling continued to hold. Baker describes scaling laws as precisely measured empirical observations whose underlying mechanisms remain poorly understood. He argues that reasoning bridged an approximately 18-month hardware gap when larger coherent Hopper clusters were impractical and Blackwell systems were delayed by deployment complexity.

• AI progress now combines pre-training, reinforcement learning with verified rewards, and test-time compute. These mechanisms are described as multiplicative, allowing improved reasoning techniques to operate on stronger base models. Google gained a temporary infrastructure advantage through newer TPUs, while Nvidia's Blackwell transition required liquid cooling, heavier racks, far more power, and difficult heat management.