What is Needed to Fully Unlock Self-Driving Cars?

TL;DR

Fully unlocking self-driving cars requires generalizing across both tight, low-speed traffic and faster multi-lane streets, while building more realistic simulation. The discussion highlights 45-mile-per-hour Phoenix roads, billions of simulated miles in good weather, and a mix of general and specialized deep-learning models. Read on to see how rain, fog, pedestrian intent, live processing, and data infrastructure shape the remaining challenge.

Transcript

are there any important technological unlocks that you still see on the horizon that not just waymo but the you know the industry of autonomy is still trying to solve think of a driver that's capable of this kind of tight traffic navigation yes lots of pedestrians and cyclists but low speed of travel that's where we are right now yes now imagine in... Read More

Key Insights

• 🚄 Navigating tight traffic and high-speed streets are crucial challenges for autonomous driving technology.
• ☀️ Improving simulation capabilities for different weather conditions and complex scenarios is essential for testing and learning.
• ❓ Generalizable AI models and specialized models are used at different layers of the autonomous driving system.
• ❓ Understanding human behavior and intents is a challenging task that requires specific models.
• 🖐️ Google's infrastructure and machine learning investments play a significant role in the development of autonomous driving algorithms.
• 🫒 Reacting in milliseconds and processing live data are essential for safe and efficient autonomous driving.
• 🕵️ Autonomous vehicles are capable of detecting and sensing the presence of various objects and obstacles on the road.

Questions & Answers

Q: What technological advances are still needed to fully unlock self-driving cars?

The industry still needs to generalize driving systems across two demanding settings: tight, low-speed traffic with pedestrians and cyclists, and faster three- or four-lane streets with heavy oncoming traffic. It also needs more realistic simulation that can combine rain, terrain, cyclists, and previously encountered difficult situations.

Q: Why are tight traffic and high-speed streets both important for autonomous driving?

Tight traffic requires navigating around many pedestrians and cyclists at low speeds, while places such as Phoenix can involve 45-mile-per-hour travel on three- or four-lane streets with substantial oncoming traffic. The speaker says good-weather cities can largely be viewed as combinations of these two driving environments.

Q: How would solving the San Francisco- and Phoenix-style driving challenges help?

San Francisco-style driving represents dense, slower traffic with pedestrians and cyclists, while Phoenix-style driving represents faster boulevards and multi-lane roads. Solving both would make the AI much more generalizable across cities, including Los Angeles areas that combine these patterns.

Q: Why does autonomous-driving simulation need to improve?

Waymo has billions of simulated miles in good weather, but the simulator must realistically reproduce other conditions. Better simulation could apply rain or terrain to difficult situations learned in good weather and add complications such as a cyclist.

Q: Does autonomous driving use one algorithm or multiple AI models?

It uses many deep-learning models rather than one single algorithm. Some models are highly general, while specialized models address difficult tasks such as interpreting pedestrian intent or driving politely and comfortably for riders.

Q: How does an autonomous-driving system respond to dense fog?

The system considers both observations of how other people drive in fog and its own ability to see ahead. As fog becomes denser and visibility falls, it reasons that it should not drive as fast as it normally would, with that learning built into multiple layers of the stack.

Q: Where is AI used within the autonomous-driving stack?

AI operates at every layer described in the discussion. Its roles include perceiving the world, predicting other people's behavior, driving the vehicle, and testing the system.

Q: Why are data infrastructure and live processing important for self-driving cars?

Building a well-developed driving algorithm requires infrastructure capable of ingesting and learning from a very large amount of data. The discussion credits machine-learning investments made by Waymo and Google more than 12 or 13 years ago, while also emphasizing that vehicles must interpret and process live information and react in milliseconds.

Summary & Key Takeaways

• The industry of autonomy is working on solving technological challenges, such as navigating tight traffic with pedestrians and cyclists at low speeds, and navigating high-speed streets with heavy oncoming traffic.

• Improving simulation capabilities is crucial, as it allows for testing and learning in different weather conditions, terrains, and complex situations.

• Autonomous driving systems require a combination of generalizable AI models and specialized models to handle various tasks, such as understanding pedestrian intent and driving like a polite and comfortable passenger.