How Will Cost, Control, and Compute Shape AI?

TL;DR

AI deployment is shifting from choosing one leading model to orchestrating several models according to each task’s performance, cost, and control requirements. Companies can build evaluations around their own workloads, use cheaper open-weight models for routine work, escalate difficult tasks to frontier models, and optimize the value produced by each token relative to computing power.

Transcript

AI is entering its postfrontier era where the model is no longer the whole product. The value will be in routing cost control and compute and openw weight models. They may start to squeeze the frontier labs. The >> model alone is no longer the product. It is the harness. >> 90 plus% of the tokens created will come out of openw weight models over th... Read More

Key Insights

• The model alone is no longer the complete AI product. Practical value increasingly comes from the harness that connects models to tools, enterprise context, evaluation processes, routing logic, and the specific production workload a company needs to perform.
• Company-specific evaluation is more useful than relying exclusively on public benchmarks. DoorDash tested AI code review with its own codebase, pull requests, and engineering problems, allowing it to compare systems against the work its developers actually encounter.
• No single model won every part of DoorDash’s code-review evaluation. A basic one-pass reviewer missed substantial information, while a system that assigned different roles to multiple models produced an approach better aligned with the company’s actual requirements.
• Model routing works by matching tasks with different cost and performance profiles. Perplexity’s previewed system starts with a cheaper open model from China’s ZAI, then calls a stronger and more expensive model only when the task requires additional capability.
• Open-weight models can give enterprises more control over AI deployment. The discussion argues that businesses modernizing thousands of internal applications may benefit from models they can download, run, manage, and combine with private organizational context.
• Token value per watt is proposed as a defining AI efficiency metric. If data-center expansion is constrained by power, useful output per token and the amount of power consumed become important measures of whether AI spending produces an acceptable return.
• Perplexity’s orchestration strategy embeds escalation skills inside an open model. The open model can call a frontier model as an advisor or subagent, avoiding a standalone router that lacks detailed knowledge of the model and its agent harness.
• Open-weight models are predicted to generate more than 90% of tokens within 18 to 24 months, possibly by the end of the year. The prediction reflects expectations that affordability, control, and widespread enterprise deployment will increasingly favor open models.

Questions & Answers

Q: Why is the AI race shifting beyond model benchmarks?

The AI race is shifting because companies are now far enough into deployment to care about operating realities that public benchmark rankings do not fully capture. They need to understand cost, data control, compute requirements, and performance on their own workloads. As customers build internal evaluations, they can choose models based on practical results instead of accepting claims made by frontier laboratories.

Q: How should companies evaluate AI models for real workloads?

Companies should create evaluation harnesses based on their own data, tasks, customers, and operational requirements. DoorDash demonstrated this approach by testing AI code review with its codebase, pull requests, and real engineering problems. Such evaluations reveal how public benchmark results translate into company-specific value and support repeated decisions about performance, cost, and return on AI spending.

Q: What did DoorDash learn from testing AI code review?

DoorDash found that no single model performed best across every part of its code-review evaluation. A basic one-pass AI reviewer missed substantial information. The stronger approach was a system that orchestrated multiple models in different roles around the actual engineering job. The result supports treating AI as a coordinated system instead of selecting one model as a universal solution.

Q: How does AI model routing reduce deployment costs?

Model routing reduces costs by assigning routine or simpler work to cheaper models and escalating only the tasks that require greater capability. Perplexity previewed a system that begins with a cheaper open model from China’s ZAI and calls a stronger, more expensive model when needed. This avoids using the highest-cost option for every request while preserving access to frontier performance.

Q: Why are agent harnesses becoming more important?

Agent harnesses are becoming more important because different models have distinct capabilities and cost-performance trade-offs. A harness can connect models with tools, enterprise context, routing logic, and specialized roles. According to Aravind Srinivas, this integrated system becomes the product because it can select suitable resources for each task and deliver unique value using information held inside the enterprise.

Q: What does token value per watt mean for AI systems?

Token value per watt describes an effort to maximize useful results from token spending while minimizing the power consumed to produce them. Srinivas argues that this matters when data-center expansion is limited by power. Under that constraint, the important outcome is not simply generating more tokens, but producing stronger returns for each user with efficient use of compute and electricity.

Q: How can open and frontier models work together?

An open model can handle the main workflow inside an agent harness and escalate harder problems to a frontier model acting as an advisor, subagent, or tool. Perplexity is working to embed this routing skill into the open model’s weights. The approach uses whichever model is best for a task instead of forcing enterprises to choose exclusively between open and frontier systems.

Q: Why might enterprises prefer open-weight AI models?

Enterprises may prefer open-weight models because they can download, run, manage, and control them while modernizing large numbers of internal applications. Local or hybrid execution can also let an organization run an orchestrator and harness on a device it owns without paying for tokens consumed there. The discussion connects these benefits with affordability, data control, and broader distribution of AI capabilities.

Summary & Key Takeaways

• The AI race is entering a postfrontier phase in which model rankings remain relevant but no longer determine the best enterprise solution. Deployment decisions increasingly depend on real workloads, operating costs, data control, available compute, and the ability to combine specialized models with tools, context, and orchestration systems.

• DoorDash illustrates the shift from public benchmarks to company-specific evaluation. Its test used its own codebase, pull requests, and engineering problems, finding that no model won every category and that a basic one-pass reviewer missed substantial information. A coordinated, multimodel system better matched the actual code-review task.

• Perplexity is developing an orchestrator that begins with a cheaper open model and escalates to a stronger, more expensive frontier model when necessary. Aravind Srinivas argues that the model and agent harness must be treated as an integrated system, potentially supporting hybrid compute, local execution, and greater enterprise control.