What Are OpenAI and Broadcom Building Together?, The OpenAI Podcast Ep. 8

TL;DR

OpenAI and Broadcom are building a custom chip and full computing system tailored to OpenAI’s inference workloads. After about 18 months of chip-design collaboration, they plan to start deploying 10 gigawatts of racks, systems, and chips late next year, on top of OpenAI’s existing infrastructure partnerships. Read on to understand the full-stack design, expected efficiency gains, and why even this capacity may not satisfy demand.

Transcript

Andrew Mayne: Hello, I'm Andrew Mayne, and welcome to the OpenAI podcast. Andrew Mayne: Today, we're excited to be breaking some news involving Broadcom and OpenAI. Andrew Mayne: Joining me from OpenAI is Sam Altman and Greg Brockman, Andrew Mayne: and from Broadcom: Hock Tan and Charlie Kawwas. Sam Altman: A lot of ways that you would look at the ... Read More

Key Insights

• OpenAI and Broadcom have spent roughly 18 months designing a custom chip specifically for OpenAI's workloads. The partnership later expanded beyond silicon into an entire computing system because supporting large-scale inference requires coordinated design across multiple hardware and software layers.
• The planned deployment will begin late next year and ultimately add 10 gigawatts of racks, systems, and custom chips. This capacity is incremental, meaning it will supplement OpenAI's existing work with other data centers, silicon providers, and infrastructure partners.
• Vertical integration is central to the partnership because the system can be optimized from transistor fabrication through chips, racks, networking, algorithms, and the final token delivered to a user. Sam Altman expects these coordinated improvements to produce faster performance and lower model costs.
• Inference demand can grow faster than efficiency gains because improved performance and lower prices encourage substantially more usage. Sam Altman illustrates this dynamic by saying that a 10-fold optimization can be followed by 20-fold demand, leaving the need for additional capacity unresolved.
• OpenAI's models are already assisting with chip design by discovering optimizations and producing substantial area reductions. Human experts can understand the resulting improvements, but the models help surface many ideas sooner than designers could evaluate them through conventional schedules.
• Broadcom contributes semiconductor, accelerator, and system-design expertise, while OpenAI contributes detailed knowledge of frontier-model workloads. The collaboration allows both companies to customize the platform from the workload down to the transistor instead of treating the chip as an isolated component.
• ChatGPT is evolving from an interactive conversation tool into an agent that can perform work behind the scenes. Pulse demonstrates this direction by preparing personalized information related to a user's interests, but compute limitations currently restrict its availability to the pro tier.
• The long-term compute goal is to support continuously operating personal agents that help people achieve their goals. Greg Brockman says an ideal system would give everyone dedicated acceleration and compute, while acknowledging that present manufacturing capacity is nowhere near that scale.

Questions & Answers

Q: What are OpenAI and Broadcom building together?

They are designing a custom chip for OpenAI’s workloads and a complete computing system to support it. The collaboration covers the chip, racks, networking, algorithms, and the product that ultimately delivers a token to the user.

Q: How long have OpenAI and Broadcom worked on the custom chip?

Sam Altman says the companies have worked together for about 18 months on the new custom chip. They more recently expanded the collaboration to include an entire custom system as the project became more complex.

Q: When will OpenAI and Broadcom begin deploying the new AI systems?

Sam Altman says deployment will begin late next year. The plan calls for 10 gigawatts of racks containing the jointly developed systems and chip.

Q: Why is OpenAI developing a chip for inference workloads?

OpenAI concluded that the world would need a large amount of inference capacity. A chip designed for that specific workload can be coordinated with the rest of the system to increase the capacity OpenAI can offer through its services.

Q: What does full-system design mean in the OpenAI and Broadcom partnership?

The partners are optimizing the infrastructure from etched transistors through chips, racks, networking, algorithms, and the token returned by ChatGPT. Sam Altman says optimizing across that entire stack can deliver efficiency gains, better performance, faster models, and cheaper models.

Q: Is the planned 10 gigawatts replacing OpenAI’s other infrastructure projects?

No. Sam Altman describes the 10 gigawatts as incremental capacity on top of OpenAI’s work with other partners, data centers, and silicon partnerships.

Q: Why might 10 gigawatts of additional capacity still not meet demand?

Sam Altman says better performance and cheaper, smarter models cause people to use much more intelligence. He gives the example that a 10-fold optimization can be followed by 20-fold demand, so efficiency improvements do not necessarily eliminate the need for more capacity.

Q: What could OpenAI’s additional computing capacity be used for?

Sam Altman names writing code, automating more enterprise work, and generating videos in Sora as current uses that could expand. The added capacity is also intended to let people do more of those tasks with smarter models.

Summary & Key Takeaways

• OpenAI and Broadcom announced a partnership built on roughly 18 months of work designing a custom chip for OpenAI's workloads. The collaboration expanded into a complete computing system because chip architecture, racks, networking, algorithms, and products must be optimized together to deliver the required inference capacity and efficiency.

• The partners plan to begin deploying racks late next year and ultimately add 10 gigawatts of computing capacity beyond OpenAI's existing infrastructure efforts. Sam Altman expects that faster, cheaper, and more capable intelligence will quickly create additional demand across coding, enterprise automation, Sora video generation, and other applications.

• OpenAI is applying its own models to chip design, producing area reductions and accelerating optimizations that human experts might otherwise reach later. The broader goal is to supply enough compute for proactive agents that work continuously, including personalized ChatGPT experiences, although current capacity limits some features to the pro tier.