How Do Recursive Language Models Use Agents?

Transcript

Oh, like I love cloud code. I love codeex. I love pi. Like no, I love oh my pi. I love prime agent. To be honest, I think all of them are the same. >> You know, your your models actually are capable a lot more if you try harder. So this is a skill issue. >> Yeah. Yeah. Basically, I I I mean I I I think there's one thing I want to see. I I appreciat... Read More

Key Insights

• Current coding agents are more similar than their branding suggests because much of their apparent differentiation comes from the surrounding harness rather than wholly distinct underlying capabilities. Zhang believes stronger system design could expose abilities that frontier models already possess but primitive agent wrappers fail to use consistently.
• GPU Mode is a community for learning and practicing GPU kernel programming through accessible lectures, discussions, leaderboards, and competitions. Its beginner-friendly character distinguishes it from benchmarking efforts where serious participation is largely concentrated among established laboratories and companies with substantial specialized resources.
• GPU kernel optimization has a relatively constrained space of commonly used techniques and frequently optimized operations. This structure encouraged the GPU Mode community to investigate whether enough examples and competitive problems could support scalable automation of kernel development through language models.
• AI-generated GPU kernels have a verification problem because benchmark success does not guarantee stability in complete systems. Zhang notes that reward hacking has been recognized since KernelBench, and that a top expert-guided solution was unusual among leading entries for remaining stable under end-to-end use.
• Human expertise is valuable because knowledgeable practitioners can steer models toward promising solutions and recognize flawed outputs. One strong insight from an expert may eliminate a vast amount of blind search that would otherwise consume hundreds of billions or even a trillion tokens.
• Large agent swarms work by spending substantial compute to explore many possible solutions. The discussion cites an experiment involving 10,000 agents, 88 hours, and 130 billion public tokens, with an estimated public price of about $40 million.
• Recursive language models extend model operation through mechanisms such as context offloading and programmatic subagent calls. The broader research direction includes persistent subagents and harnesses that compose multiple capabilities, potentially presenting users with one simple interface backed by many cooperating agents.
• Long-term reliability is still an unresolved limitation because current frontier models are not consistently good enough to perform a particular job well across a month. Massive compute creates new options for solving difficult problems, but it does not automatically provide efficient, dependable, or well-verified execution.

Questions & Answers

Q: What is a recursive language model?

A recursive language model is presented as a system that expands how a model handles difficult work through context offloading and programmatic calls to subagents. Instead of requiring one model invocation to retain and process everything directly, the surrounding harness can delegate portions of the task, preserve useful working context, and combine the results of multiple agent operations.

Q: How can agent harnesses improve language model capabilities?

Agent harnesses can improve capability by organizing context, tools, subagent calls, and repeated work around a model. Zhang’s central concern is that increasingly powerful models are still wrapped in comparatively primitive systems. Better harness design may reveal capabilities already latent in current models by enabling decomposition, persistent work, compositional behavior, and more deliberate verification of intermediate results.

Q: Why are Claude Code, Codex, and other coding agents similar?

Zhang says he likes several coding agents but regards them as broadly the same. The discussion suggests that important differences often lie in the harness, workflow, and interface surrounding increasingly capable models. This shifts attention from treating each product as fundamentally unique toward studying how context management, tool use, subagent coordination, and verification determine the system’s practical performance.

Q: Why do AI-generated GPU kernels still need human experts?

AI-generated kernels still need experts because high benchmark placement does not ensure that a kernel is correct, stable, concise, or useful inside a complete system. Experienced kernel writers can identify promising optimization directions, detect reward hacking, and act as strong verifiers. Zhang describes an expert-guided kernel as the only leading solution that was stable in actual end-to-end systems.

Q: What is the verification problem in GPU kernel benchmarks?

The verification problem is the gap between obtaining a strong benchmark score and producing a kernel that behaves reliably in real end-to-end use. Zhang says GPU kernels have long faced reward hacking, including in work associated with KernelBench. A generated solution can exploit evaluation weaknesses or appear fast under limited tests while remaining unstable, overly complex, or unsuitable for actual deployment.

Q: How does expert knowledge reduce the cost of agent search?

Expert knowledge reduces search cost by steering models toward productive approaches and rejecting weak ones before excessive compute is spent. Zhang observes that enough agents and tokens can explore a problem extensively, but a person who understands the domain may supply an insight that makes an enormous search unnecessary. He illustrates the contrast with searches consuming hundreds of billions or even a trillion tokens.

Q: What did the 10,000-agent experiment demonstrate?

The cited experiment used 10,000 agents over 88 hours and consumed 130 billion public tokens, estimated at about $40 million in public pricing. It demonstrated that sufficiently large computational budgets can be directed at exploring and solving a single problem. The discussion treats that possibility as exciting, while also recognizing that such brute-force expenditure is not an affordable default for every problem.

Q: What limits current frontier models on long-running work?

Current frontier models remain insufficiently consistent when asked to perform a particular job well across a period as long as a month. The limitation is not simply whether a model can complete an isolated impressive task. Reliable long-running work also requires stable planning, context management, verification, recovery from errors, and a harness that coordinates repeated activity without allowing failures to accumulate.

Summary & Key Takeaways

• Alex Zhang traces his path from GPU Mode and kernel optimization to research on recursive language models. GPU Mode began as a community for learning GPU kernel programming, offering accessible lectures and competitions. Its projects explored whether language models could automate specialized kernel development, an important bottleneck for making new research systems practically usable.

• AI-generated kernels can rank highly on benchmarks while failing under end-to-end conditions because kernel evaluation has a verification and reward-hacking problem. Zhang highlights a leading community member whose expert steering produced unusually stable, concise solutions. Domain knowledge therefore remains valuable as a source of direction, judgment, and reliable verification when models explore possible implementations.

• Recursive language models, context offloading, programmatic subagent calls, and persistent agents represent attempts to build stronger systems around existing models. Zhang views harnesses as compositional generalizers and imagines future language-model interfaces backed by invisible agent swarms. Massive experiments show that compute can explore solutions broadly, but efficiency and month-long consistency remain unresolved challenges.