Drago Anguelov (Waymo) - MIT Self-Driving Cars | Summary and Q&A

TL;DR

Weymo's Drago Anguelov discusses the challenges of self-driving cars and the need to tackle the long tail of rare and complex scenarios through machine learning and expert-designed algorithms.

Key Insights

• 😀 Weymo has achieved over 10 million miles of autonomous driving and is focused on addressing the long tail of rare and complex scenarios.
• 😨 The challenges of self-driving cars include perception, prediction, and planning, which require handling diverse objects, environments, and configurations.
• 🎰 Machine learning is a crucial tool for developing scalable models, but it needs to be complemented with expert-designed algorithms to ensure safety and reliability.
• 🍵 Weymo utilizes redundant and complementary sensors, as well as hybrid systems, to improve robustness and handle uncertainty in real-world driving.
• 🌍 Testing and validation are conducted through a combination of real-world driving, structured testing, and simulation, with a focus on scalability and safety.
• ☢️ Weymo addresses the challenges of uncertainty and limited data through active learning, data mining, transfer learning, and reinforcement learning techniques.
• 🖐️ Reasoning and expert-designed algorithms play an important role in tackling complex scenarios and improving the performance of self-driving systems.

Transcript

all right welcome back to 6sz ro9 for deep learning for self-driving cars today we have Drago and glial of principal scientists at way mo aside from having the coolest name in autonomous driving Drago has done a lot of excellent work in developing applying machine learning methods to autonomous vehicle perception and more generally in computer visi... Read More

Questions & Answers

Q: How does Weymo handle the diverse range of scenarios that self-driving cars may encounter?

Weymo collects data from real-world driving and uses machine learning to develop models that can handle a wide range of objects, environments, and complex scenarios. They also use expert-designed algorithms to address limitations and ensure safety.

Q: How does Weymo ensure the safety and reliability of their self-driving cars?

Weymo utilizes redundant and complementary sensors, such as cameras, lidar, and radar, to gather data from different perspectives and modes of failure. They also use hybrid systems that combine expert-designed algorithms and machine learning models to improve robustness in uncertain situations.

Q: How does Weymo handle testing and validation of their self-driving algorithms?

Weymo uses a combination of real-world driving, structured testing in controlled environments, and simulation. They have a rigorous testing process and a scalable simulation infrastructure that allows them to test a large number of scenarios and ensure the safety and performance of their algorithms.

Q: How does Weymo address the challenges of uncertainty and limited data in training their models?

Weymo leverages active learning and data mining techniques to select interesting and rare scenarios for labeling and training. They also utilize transfer learning and reinforcement learning to improve the performance of their models. Additionally, they use domain knowledge and expert-designed models to handle situations with limited data.

Q: How does Weymo handle the diverse range of scenarios that self-driving cars may encounter?

Weymo collects data from real-world driving and uses machine learning to develop models that can handle a wide range of objects, environments, and complex scenarios. They also use expert-designed algorithms to address limitations and ensure safety.

More Insights

• Weymo has achieved over 10 million miles of autonomous driving and is focused on addressing the long tail of rare and complex scenarios.

• The challenges of self-driving cars include perception, prediction, and planning, which require handling diverse objects, environments, and configurations.

• Machine learning is a crucial tool for developing scalable models, but it needs to be complemented with expert-designed algorithms to ensure safety and reliability.

• Weymo utilizes redundant and complementary sensors, as well as hybrid systems, to improve robustness and handle uncertainty in real-world driving.

• Testing and validation are conducted through a combination of real-world driving, structured testing, and simulation, with a focus on scalability and safety.

• Weymo addresses the challenges of uncertainty and limited data through active learning, data mining, transfer learning, and reinforcement learning techniques.

• Reasoning and expert-designed algorithms play an important role in tackling complex scenarios and improving the performance of self-driving systems.

• The future of self-driving cars involves scaling to different environments and continuously improving the models and infrastructure to handle the long tail of challenging scenarios.

Summary

In this video, Drago Anguelov, principal scientist at Waymo, discusses the challenges and solutions in self-driving cars. He talks about the diverse scenarios and situations that self-driving vehicles need to handle, which he calls the "long tail" of autonomous driving challenges. He emphasizes the importance of perception, prediction, and planning in self-driving algorithms and how machine learning is used to tackle these challenges. He also discusses the role of infrastructure, high-quality label data, and models in the machine learning process. Finally, he highlights the importance of large-scale testing and the use of simulations and agents in the development and testing of self-driving algorithms.

Questions & Answers

Q: What is the "long tail" of autonomous driving challenges?

The "long tail" refers to the diverse and rare scenarios and situations that self-driving vehicles need to handle in order to achieve truly autonomous driving. These situations, although not common, are important to handle well as they can have significant impact on safety and performance.

Q: How is Waymo approaching the long tail of autonomous driving challenges?

Waymo is using machine learning to address the long tail of autonomous driving challenges. They collect large amounts of data from their vehicles, both in real-world and simulated scenarios. They leverage their machine learning infrastructure to train models on this data and continuously improve their algorithms. They also use a hybrid approach, incorporating expert domain knowledge and designing input distributions that are easier to learn with fewer examples.

Q: What are the core AI aspects of self-driving cars?

The core AI aspects of self-driving cars are perception, prediction, and planning. Perception involves mapping sensory inputs and prior knowledge of the environment to a meaningful representation, which includes objects, semantics, and relationships. Prediction involves anticipating and predicting the behavior of other agents in the environment, such as pedestrians and other vehicles. Planning is responsible for decision-making and generating behavior that is safe, comfortable, and efficient.

Q: How does Waymo use machine learning in their self-driving algorithms?

Waymo uses machine learning throughout their self-driving stack. They utilize deep learning frameworks like TensorFlow and have access to specialized hardware for training models. They have a well-developed machine learning infrastructure that allows them to train models on large-scale data. They also collaborate with other research teams within Alphabet, such as Google and DeepMind, to improve their models and algorithms.

Q: How does Waymo approach large-scale testing and simulations?

Waymo utilizes large-scale testing and simulations as part of their development and testing process. They have a simulator that simulates the equivalent of 25,000 virtual cars driving 10 million miles per day. They use structured testing, where they deliberately stage and test specific scenarios. They also leverage their driving data to select interesting situations and create variations of those situations to generate more scenarios for testing. Simulations allow them to test and validate their algorithms in a wide range of conditions and situations.

Q: How does Waymo handle uncertain or rare situations in their self-driving algorithms?

Waymo ensures robustness in their self-driving algorithms by leveraging redundant and complementary sensors, such as cameras, lidar, and radar. This helps to ensure that they don't miss any important information and allows them to handle different modes of failure. They also design their system to be a hybrid, incorporating expert domain knowledge and designing input distributions that are easier to learn with few examples. This helps in situations where the machine learning system may not be confident or may make mistakes.

Q: How does Waymo train their self-driving models?

Waymo trains their self-driving models using a large amount of high-quality labeled data. They collect and store data from their vehicles, and then select interesting parts of the data to send to labelers for annotation. They also use active learning and data mining pipelines to find and label rare examples and cases that the models may not be handling well. They also produce auto labels by leveraging their perception system's understanding of the past and using that knowledge to annotate the data for training.

Q: Can you explain the concept of "machine learning factory" in self-driving models?

The "machine learning factory" in self-driving models refers to the process of training and iterating models using a large-scale, automated infrastructure. Waymo collects data from their vehicles, sends parts of the data to labelers for annotation, and then trains machine learning models on this labeled data. They have a scalable setup that allows them to iterate and improve their models continually. This infrastructure includes computing software infrastructure, high-quality label data, and high-quality models.

Q: How does Waymo handle testing and validation of their self-driving algorithms?

Waymo performs testing and validation of their self-driving algorithms through a combination of structured testing, simulations, and real-world driving. They have test environments, such as an air force base, where they can deliberately stage and test specific scenarios. They also utilize simulations to simulate a large number of virtual cars driving in various conditions. They ensure that their algorithms are well-tested before deploying them on public roads using a combination of real-world and simulated testing.

Q: How does Waymo handle the behavioral aspect of self-driving algorithms?

Waymo handles the behavioral aspect of self-driving algorithms by using agents that imitate and learn from real-world driving. They use deep neural networks to imitate driver behavior based on large amounts of driving data. They also add constraints and considerations, such as collisions, staying on the road, and avoiding unsafe actions, to their training process. This helps to ensure that the learned policies are realistic and safe. They also use expert design and models as part of their hybrid approach to handle complex and interactive situations.

Takeaways

Waymo is using machine learning to tackle the long tail of autonomous driving challenges, which involve diverse and rare scenarios that self-driving cars need to handle. They have a strong focus on perception, prediction, and planning in their self-driving algorithms. Waymo leverages their machine learning infrastructure, high-quality label data, and expert domain knowledge in the training of their self-driving models. They also utilize large-scale testing, simulations, and the use of agents to develop and test their self-driving algorithms. Overall, Waymo is committed to developing and advancing autonomous driving technology to make it safe and reliable for the real world.

Summary & Key Takeaways

• Weymo has achieved over 10 million miles of autonomous driving and is focused on tackling the long tail of rare and challenging scenarios.

• The challenges of self-driving cars include perception, prediction, and planning, which requires the handling of diverse objects, environments, and configurations.

• Weymo leverages machine learning to develop scalable models and uses expert-designed algorithms to address the limitations of current machine learning technology.