Dean Leitersdorf on AI-Generated Video Games and Worlds

TL;DR

Decart's Oasis is the first video model that runs in real time and lets users interact with a playable game world with no game engine, moving around and placing or breaking blocks. Its founder frames the bigger goal as overcoming the communication barrier between humans and computers through generative interactive video, connecting what we imagine to what we see on screen.

Transcript

so we we we launched the ois a few weeks ago and really you know when we launch it the the incredible thing from a tech perspective was oh this is the first video model that actually runs real time and you can interact with it in response to user actions you can move around the world you can break blocks you can place blocks and so we we got this t... Read More

Key Insights

• Oasis is described as the first video model that runs in real time and lets users interact with it in response to their actions, moving around a world and breaking or placing blocks, producing a game without a game engine.
• The 'magical mirror' analogy captures the vision: users could talk to a screen, ask for a sword in their hand, resize or recolor it, turn the scene into Game of Thrones, and have the image respond live to movement.
• Generative interactive video is interesting because it is a completely different experience than anything before, letting people channel imagination through screens by connecting what they see in their minds to what they see with their eyes.
• ChatGPT does not solve any single problem but overcomes a fundamental limitation, the communication barrier between humans and computers, because computers speak structured languages and humans speak unstructured ones that LLMs bridge.
• Interactive video extends that same barrier-breaking beyond text to vision, letting computers see the world the way humans do and show the world back in ways humans understand, forming a platform others build on.
• Overcoming a limitation rather than solving a problem is rare: 99% of companies solve problems, while chances to build something that overcomes a limitation come maybe once a decade or every 15 years.
• Game engines exist so one person can create a world and another can interact with it, offering consistency and accuracy but requiring a lot of time to work with.
• Because an AI runs the environment, users can interact with it the way they interact with AI, saying things like make it Elsa-themed or add a flying elephant, and the elephant is interactive rather than just a picture.

Questions & Answers

Q: What is Decart's Oasis?

Oasis is a fully playable AI game engine that Decart launched a few weeks before the interview. From a technical perspective, it is described as the first video model that actually runs in real time and lets you interact with it in response to your actions. You can move around the world, break blocks, and place blocks, producing a game without a traditional game engine. The founder frames Oasis one as an early step toward much larger interactive video experiences.

Q: How does Dean Leitersdorf explain the vision using a mirror?

He asks you to imagine a magical mirror you can talk to. You could tell it you are there and want to hold a sword, and a sword appears in the mirror where your hand is, moving as your hand moves. You could ask to make the sword bigger or blue, turn everything around you into Game of Thrones, get a crown, and start jumping, with the mirror responding to all of it. This connects what we see in our minds to what we see with our eyes.

Q: Why is generative interactive video considered important?

It is important because it is a completely different experience than anything we have had before on Earth. It allows us to channel our imagination through screens we can see, connecting two things: what we see in our minds and what we can see with our eyes. In a single sentence, the aim is how generative AI can let us connect our imagination to what appears on our screens, taking us into worlds we have not explored and even changing how we interact with computers and hardware.

Q: What is the difference between solving a problem and overcoming a limitation?

The founder argues that ChatGPT does not solve any single given problem; it helps with homework, writing emails, and summarizing, but really it overcomes a fundamental limitation, the communication barrier between humans and computers. Computers speak in structured languages and humans in unstructured or complex ones, and LLMs bridge that gap. Overcoming a limitation opens a hundred things solved on top of it, whereas 99% of companies simply solve a defined problem.

Q: How does interactive video extend what ChatGPT started?

ChatGPT overcame the communication barrier between humans and computers using text. Interactive video, or the mirror concept, overcomes that same barrier with what we can see. Computers will be able to see the world the way we see it and show us the world in ways we can understand. Solving that gives you a platform to build everything on top of it, from a next-generation Snapchat or TikTok all the way to simulators for fighter pilots.

Q: Why does Dean compare Decart's technology to the personal computer?

He asks whether the Mac is a consumer or enterprise product, hardware or software, and what problem it solves. The point is that you cannot give a single list, because the personal computer spans everything from gaming to Excel. Similarly, Decart is building a powerful piece of technology that can be productized in many different ways, rather than being tied to one problem, which is presented as a strength of overcoming a limitation.

Q: What were game engines originally meant to do?

Game engines were supposed to let one person create a world and a different person interact with that world. That is their purpose: you have the game developer and the user who uses what they made, and the same applies to movies, with Unreal being used for films recently. They are a valuable product with real advantages, making worlds very consistent and accurate, but the drawback is that they take a lot of time to interact with and build for.

Q: How does an AI-run environment change how people interact with games?

Because an AI runs your game or environment, you can interact with it the way you are used to interacting with AI. You could say turn this into something Elsa-themed and everything becomes Elsa-themed, or add a flying elephant and one appears. Crucially it is not just a picture; you can actually interact with it, such as punching the elephant and having it punch back. The founder notes that for this to replace game engines, it would need to reach a state where you can program for it.

Summary & Key Takeaways

• Decart launched Oasis, described as the first real-time interactive video model, producing a playable game world with no game engine where users move around and break or place blocks. The team stresses that the real significance lies not in Oasis one but in where the technology is heading.

• The vision is illustrated by a magical mirror: users talk to a screen, ask for a sword, resize or recolor it, transform their surroundings into Game of Thrones, and see everything respond in real time. This connects what people imagine in their minds to what they see with their eyes.

• Like ChatGPT overcoming the human-computer communication barrier rather than solving one problem, interactive video extends that bridge to vision, becoming a platform for products from next-gen Snapchat or TikTok to fighter-pilot simulators, potentially replacing game engines if it becomes programmable.