Vijay Kumar: Flying Robots | Lex Fridman Podcast #37

TL;DR

Vijay Kumar, a leading roboticist, discusses the potential and challenges of robotics, including the collaboration between humans and robots, the limitations of computer vision, and the future of autonomous vehicles.

Transcript

the following is a conversation with Vijay Kumar he's one of the top roboticist in the world a professor at the University of Pennsylvania a Dean Afeni engineering former director of grasp lab or the general robotics automation sensing in perception laboratory a pen that was established back in 1979 that's 40 years ago Vijay is perhaps best known f... Read More

Key Insights

• 🤖 Robotics: Vijay Kumar is a renowned roboticist known for his work in multi-robot systems, robot swarms, and micro aerial vehicles. He is focused on creating robots that can elegantly cooperate in flight under real-world conditions.
• 🌐 Global Impact: Vijay Kumar believes that robots should be designed and used to solve real-world problems for humans. He sees a great potential for the use of robots in various applications including last-mile delivery, disaster response, and remote healthcare, among others.
• ⚙️ Technical Challenges: One of the main challenges in robotics is coordinating multiple motors and sensors to achieve efficient and agile movement. Furthermore, modeling and predicting the behavior of complex real-world environments is difficult but crucial for successful autonomous navigation.
• 🧠 Collaboration with Humans: Vijay Kumar emphasizes the importance of human-robot collaboration in various tasks. He believes that robots should be able to understand and work with humans in different contexts, whether through collaboration, command, or as bystanders.
• 💡 Future of Robotics: One of the key goals in robotics is to develop robots that can operate in unstructured environments and adapt to changing conditions. This requires advancements in perception, planning, and control algorithms, as well as improvements in energy storage and power consumption.
• 🚗 Autonomous Driving: While autonomous driving has made significant progress, there are still challenges ahead, particularly in terms of ensuring safety and solving complex corner cases. Vijay Kumar believes that a combination of machine learning and model-based approaches will be essential for achieving high levels of autonomy in driving.
• 🔬 Open Problems: Some of the current open problems in robotics include developing robots that can operate in various settings and adapt to changing environments, as well as finding the optimal balance between machine learning and model-based approaches for different tasks.
• 💭 Ethical Considerations: Although there are concerns about the potential misuse of robotics technology, Vijay Kumar believes that it is important for engineers and policymakers to shape and guide the development of robotics. He emphasizes the need for ethical and responsible use of robotics technology for the benefit of society.
• 🎓 Advice for Students: Vijay Kumar suggests that students interested in robotics should have a strong background in mathematics, as well as a broad understanding of other disciplines such as liberal arts. He also encourages them to think about the future and to remain adaptable and open to new ideas and possibilities.

Questions & Answers

Q: How does Vijay Kumar see the role of humans in collaborating with robots in different scenarios?

Kumar believes that human-robot collaboration is crucial in solving complex tasks and ensuring safety, and that engineers and politicians should work together to understand and shape the future of robotics.

Q: What are the challenges in achieving autonomy in flying robots and autonomous vehicles?

Kumar highlights the difficulties in modeling and navigating in 3D environments, as well as the limitations of current battery technology for achieving efficient and long-range flight in drones and flying cars.

Q: How does Vijay Kumar view the potential risks associated with robotics, like weaponizing robots or the implications of advancing AI technology?

Kumar acknowledges the potential risks and advocates for engineers and researchers to be aware of such risks, prioritize safety, and work towards building technology for beneficial purposes.

Q: What advice does Vijay Kumar have for aspiring robotics and AI students?

Kumar advises students to keep up with the rapidly changing field, embrace uncertainty, develop a broad understanding of various disciplines, and place importance on mathematical foundations and representation in robotics.

Summary

This conversation is with Vijay Kumar, a renowned roboticist and professor at the University of Pennsylvania. They discuss various topics related to robotics, including the first robot Kumar ever built, the challenges of coordinating motors in a robot, the beauty of robot swarms, the role of machine learning in robotics, the limits of computer vision, the future of autonomous vehicles, the potential of delivery drones, the challenges of battery technology in flying robots, and the possibility of flying cars.

Questions & Answers

Q: What was the first robot Kumar ever built?

Kumar built a large hexapod robot during his time in graduate school. The robot had 18 joints controlled by independent computers, with a 19th computer for coordinating the joints' movements.

Q: How difficult is it to make all the motors in a robot communicate with each other?

It was challenging to make the motors communicate in the past when Kumar was working on his first robot. They had to develop a multiprocessor operating system to ensure messages were transmitted between the joints. The computers' clock speeds were about half a megahertz, making the task more complicated.

Q: How did Kumar feel when he saw the robot move for the first time?

Kumar describes it as amazing to see the robot move. Although in hindsight, he acknowledges that the robot was unnecessarily large compared to today's robots. However, the size gave it a grandeur that was appreciated during that time.

Q: What is the most beautiful or elegant motion of a robot that Kumar has seen?

Kumar is most proud of his students' work on small unmanned aerial vehicles (UAVs) that can coordinate and form three-dimensional patterns in flight. This ability allows them to create and deform objects in the sky, enhancing their capabilities.

Q: What living creatures have inspired Kumar's work in robotics?

Kumar finds ants to be incredibly inspiring due to their resilience and ability to adapt as a population. He admires their instinct for self-preservation, their ability to work together without direct communication, and their demonstration of consensus and democracy as a colony.

Q: Is it possible to consider biological swarms or robot swarms as a single intelligent organism?

From an engineering perspective, the goal is to think of the swarm as a cohesive unit without focusing too much on the individual components. By abstracting the individuals, the swarm can be perceived as a powerful and synergistic whole.

Q: What does it take for a group of flying robots to form a formation?

The ability for a group of flying robots to form a formation involves local sensing, coordination, and a global reference frame. The robots need to know who their immediate neighbors are, be cognizant of the global environment, and estimate their position with respect to the global reference frame.

Q: How challenging is it to make a small quadcopter fly?

Making a small quadcopter fly involves coordinating the motors to generate the right amount of thrust, positioning the quadcopter to fly in the desired direction, and estimating its position and velocity. The challenge lies in achieving smooth and efficient control while aligning with safety constraints.

Q: Can machine learning play a role in robotics, specifically in flying robots?

Machine learning can play a role in robotics, especially in perception tasks such as computer vision. However, for action-based tasks, there are limited successful examples of machine learning being used. Learning-based approaches, in combination with model-based approaches, may be the future direction.

Q: What are the limits of computer vision in autonomous vehicles?

While computer vision has made significant advancements in object recognition and classification, there are limits to its effectiveness. Certain environments, such as dark or dirty areas, can pose challenges for computer vision algorithms. Additionally, there are corner cases that may not be effectively addressed by relying solely on computer vision.

Q: Which problem is harder to solve, autonomous driving or autonomous flight?

Autonomous flight has some advantages over autonomous driving, such as the ability to plan vertically and have safer trajectories. However, autonomous flight also requires dealing with aerodynamic challenges and modeling a three-dimensional world. Both autonomous driving and autonomous flight present their own unique difficulties.

Q: Is it possible to envision a future with tens of thousands or hundreds of thousands of delivery drones filling the sky?

There is potential for delivery drones to be widely used, especially in crowded cities or remote areas. Delivery drones can significantly reduce travel time and transportation challenges. However, the primary challenge is the technological and economic feasibility, including optimizing energy usage and solving battery-related issues.

Q: What are the challenges of flying cars, and is there a possibility of realizing this dream?

Flying cars present several challenges, including safety concerns, energy requirements, and economic viability. While flying cars are a topic of interest and research, it remains to be seen if the vision of a widespread fleet of affordable flying cars can be realized.

Takeaways

Robotics has come a long way in terms of advancements in technology and capabilities. Quadcopters and other flying robots have seen significant progress, but challenges remain in making them more versatile and autonomous. Coordination of motors, perception limitations, energy storage, and safety considerations are areas that need further exploration. Additionally, while machine learning plays a role in perception tasks, there is still room for improvement in action-based learning. The future of autonomous vehicles, delivery drones, and flying cars depends not only on technological advancements but also on addressing economic and societal needs.

Summary & Key Takeaways

• Vijay Kumar recalls his early experience in building a large hexapod robot and the challenges of coordinating multiple motors and computer systems.

• He discusses the advantages and beauty of small UAVs that can coordinate with each other in 3D space, as well as the inspiration he draws from biological systems like ants.

• Kumar explores the potential of machine learning in robotics, the limitations of computer vision, and the importance of human-robot collaboration in various tasks and settings.