Kevin Weil on Building ChatGPT Products at OpenAI

TL;DR

The model you use today is the worst AI model you'll ever use for the rest of your life, so OpenAI builds products on the edge of what models can do, even when they only work 70% of the time. Within two or three months the next model makes them 95% capable. A tight loop between research and product is where the magic happens.

Transcript

You said today is the worst product from you today. The model the model that you use today is the worst AI model that you'll ever use for the rest of your life. Yeah. Kevin, so you know on this slide in five years as the ninth most valuable company in the world. Okay, so $300 billion in enterprise value today is measured by the markets. They have i... Read More

Key Insights

• The model you use today is the worst AI model you will ever use for the rest of your life, because models get step-change improvements every two or three months, which changes how you build products.
• Building on the edge of what models can do lets OpenAI ship products that do things no product has done before, even if they only work 70% initially and reach 95% within a few months.
• Weekly active users is the metric OpenAI goals itself by rather than monthly, because a once-a-month user signals the product is not adding much value to their life.
• Iterative deployment means releasing early and often, making many small mistakes so society collectively learns what AI is good and bad at, rather than waiting for perfection.
• A tight loop between research and product is where magic happens: user problems feed back to the research team, which fixes model weaknesses, and the product improves.
• The best advice for new users is simply to start using ChatGPT and take small risks trying new tasks, because in most cases it can do more than people expect.
• ChatGPT's usefulness has broadened far beyond early copywriting and email summarization to interpreting medical documents, setting appliance clocks from photos, and running multi-step lookups and calculations.
• True usefulness requires ChatGPT to move from answering questions to doing things in the real world proactively, which is why OpenAI focuses on connections, personalization, and memory.

Questions & Answers

Q: What does Kevin Weil mean by 'the worst AI model you'll ever use'?

Weil said the model you use today is the worst AI model you'll ever use for the rest of your life. Because models get step changes better every two or three months, today's version is always the least capable one you'll experience. He calls this simple but prophetic, arguing it changes how you build products by making you more open to building things that only kind of work, since the model will soon be great at them.

Q: Why does OpenAI track weekly active users instead of monthly?

OpenAI goals itself by weekly active users because someone who comes only once a month signals the product probably isn't adding much value to their life. Weekly usage shows ChatGPT is genuinely helping. Weil says over time it will be more natural to measure daily active users, because if OpenAI is doing its job, people will use ChatGPT daily and ideally multiple times per day for the many different things they encounter.

Q: How does OpenAI decide what products to build?

OpenAI does less mechanical work of nudging users from monthly to weekly. Instead it stays extremely close to its world-class research team, described as a nucleus of creativity and discovery, so it can build products just on the edge of what the models can do. That approach yields products doing things no one has seen before. Feedback from real user problems flows back to research, which fixes model weaknesses, and the product improves in a tight loop.

Q: What is iterative deployment and why does OpenAI use it?

Iterative deployment means releasing early and releasing often, favoring many small mistakes over waiting for perfection. Weil says it's better to make lots of small mistakes so society collectively understands what AI is good at, what it's bad at, and how to work through it. Even a product that's only 70% good at a task can be shipped, because the next model in two or three months makes it about 95% good, and users have been brought along the way.

Q: How did Kevin Weil use ChatGPT during his son's medical situation?

Weil's eight-year-old son had a minor surgery that carried a tiny chance of something more serious. When a letter arrived full of complex, scary-looking medical terms and Weil couldn't reach the doctor, who was in surgery, he took a picture and uploaded it to ChatGPT asking if he should be worried. It immediately replied that everything was completely benign. He couldn't reach the doctor for 72 hours, which would have been a very stressful wait as a parent.

Q: What is OpenAI most excited about on the horizon for ChatGPT?

Weil is most excited about ChatGPT going from just answering questions to actually doing things for you in the real world, ideally proactively, understanding what you'll need to do and queuing up actions before you even get there. That's why OpenAI focuses on adding connections into products like Google Docs and services you use every day, plus personalization and memory so ChatGPT gets to know you, your preferences, and how you like to interact with it.

Q: How valuable is OpenAI according to figures discussed in the interview?

The discussion referenced a slide projecting OpenAI as the ninth most valuable company in the world within five years. Its enterprise value is $300 billion today as measured by the markets, with the deck showing it going to $1.6 trillion. Weil recalled that when asked whether OpenAI could reach a trillion dollars, he said if he thought it was just a trillion dollars he wouldn't be nearly as excited about joining, emphasizing ambition, opportunity, and impact over market cap.

Q: Why does Kevin Weil advise people to just start using ChatGPT?

Weil's number one piece of advice is simply to start using ChatGPT, because when you use it for real tasks and take little tiny risks trying things you're unsure it can do, you realize in most cases it can. It then becomes more and more helpful over time. Since the models get step changes better every two or three months and the product constantly evolves, hands-on use is the best way to discover its expanding range of capabilities.

Summary & Key Takeaways

• OpenAI CPO Kevin Weil, about a year into the role and formerly at Instagram, tells BG2's Bill Gurley and Brad Gerstner that ChatGPT has grown faster than anything he has seen, and that the opportunity is bigger than he realized when he joined, though it was already in Sam's head.

• OpenAI measures success by weekly active users, aiming to move people toward daily and multiple-times-daily use. Weil shares uploading a photo of his son's post-surgery medical letter to ChatGPT, which immediately said it was benign, sparing him a stressful 72-hour wait to reach the doctor.

• Rather than mechanical growth tactics, OpenAI relies on a world-class research team and a tight research-product loop, building on the edge of model capability. Weil is most excited about ChatGPT moving from answering questions to proactively doing real-world tasks via connections, personalization, and memory.