Thomas Wolf on Hugging Face's LeRobot and Physical AI

TL;DR

Robotics today is at the same inflection point language models reached a few years ago: the hardware exists, but adaptive software was the missing piece. Hugging Face's LeRobot combines policy models, datasets, and affordable hardware, aiming to turn millions of software developers into roboticists the way transformers turned developers into AI researchers.

Transcript

Many many startups just already being built on top of the robot just you know they want to build something they have this idea of of a manual test they can automate or they have an idea of something they could do in the physical world and then they take the robot they take already like the basic building blocks we've shipped which is just a robotic... Read More

Key Insights

• LeRobot is Hugging Face's attempt to reproduce the success of its transformers library in robotics, combining three aspects: the policy models, the datasets, and the hardware (actuators) in one central, accessible library everyone can use.
• The missing brick in robotics was software, not hardware. Thomas Wolf argues hardware has been available for quite some time, but robotics needed software that could adapt and be dynamic, which recent breakthroughs began to deliver.
• Hugging Face started its robotics activities 18 months ago, prompted by lab breakthroughs (from teams like Stanford) showing robots that could tie knots, fold clothes, and toss and catch things on a pan, often using very little data.
• Local model hosting matters more in robotics than in LLMs for safety. If a robot loses Wi-Fi it could run into a wall or a child, so keeping models close to the hardware avoids depending on a distant API.
• The LeRobot community numbers roughly six to ten thousand people and is growing exponentially, measured by indicators like the number of datasets contributed on the hub; one worldwide hackathon spanned 100 locations across six continents.
• The developer base splits into three personas: traditional roboticists frustrated by limiting optimal-control software, AI enthusiasts drawn to robotics as a physical manifestation of AI, and curious non-technical people, including investors buying the arm to understand robotics.
• The SO-100 robotic arm was designed to be the cheapest robotic arm at $100, serving as a basic building block startups and hobbyists already use to build businesses and automate physical tasks.
• Wolf sees entertainment, fun demos, and education as the most promising near-term robotics market because reliability demands are lower there than in enterprise retail deployment, making accessible robots like the $300 Reachy Mini possible.

Questions & Answers

Q: What is Hugging Face's LeRobot project?

LeRobot is Hugging Face's attempt to reproduce the success of its transformers library in the robotics field. The idea is a central library everyone would use that brings the latest technology, algorithms, and datasets for training robots efficiently in a simple, accessible, and easy way. It connects these to actuators, the hardware part of robotics, combining three aspects: the policy models, the datasets, and the hardware.

Q: Why does Thomas Wolf think robotics is at the same moment transformers were a few years ago?

Wolf says the shift started around two years ago, with Hugging Face beginning robotics activities 18 months ago. Breakthroughs from labs, including teams around Stanford, showed robots tying knots, folding clothes, and tossing and catching things on a pan, often with very little data. This pointed to a near future where hardware was already there, and the missing brick, adaptive dynamic software, was finally emerging.

Q: Why is local model hosting more important in robotics than for language models?

Wolf explains that in a future with robots everywhere, you want many models to run locally for safety. If a robot loses connection to Wi-Fi, it could run into a wall or a child, which is far more dramatic than an LLM hallucinating. Safety concerns are a good reason to not depend on a distant API but keep the models as close as possible to the hardware, making Hugging Face's open-source role even more important.

Q: How big is the LeRobot community?

The community numbers several thousand people, roughly six to ten thousand, and is growing exponentially. One worldwide hackathon held a couple of months earlier spanned 100 locations across over six continents. It is still not in the millions, but well above several thousand. The main indicator Hugging Face tracks is the number of datasets on the hub, which shows the same exponential growth in community members and datasets.

Q: What types of developers are building in the LeRobot community?

There are three personas. First, traditional roboticists who want to use AI but were frustrated by limiting optimal-control software. Second, and most interesting to Wolf, people not originally into robotics but drawn to it as a physical manifestation of AI, including software developers. Third, curious people who are not purely technical, such as investors who bought the SO-100 arm just to physically understand what robotics can do.

Q: What does it mean to vibe code a robot?

Wolf describes robots you can control with a little bit of vibe coding because the software is just Python code, making it easy to tweak or control. He wants this to be one of the easiest ways to use the new Reachy Mini robots, and says he would love his kids to be able to vibe code behaviors on the robots. Some of this is already possible with the current tools today.

Q: How affordable is Hugging Face's robotics hardware?

Hugging Face designed the SO-100, a very simple robotic arm, specifically to be the cheapest robotic arm at $100. Startups and hobbyists already use it as a basic building block to start businesses. For consumer robots, the Reachy Mini is priced at $300, which Wolf describes as something that can be an impulsive buy or a gift, purchased without certainty it will work.

Q: Where does Thomas Wolf expect the first ChatGPT moment for robotics to happen?

Wolf is most interested in entertainment, fun demos, and education rather than enterprise. The enterprise market is complex, with heavy existing use in areas like car manufacturing and challenges around reliability for retail deployment. In entertainment and education, the demand for a $3,000 reliable robot is less pressing, so accessible robots like the $300 Reachy Mini can serve as impulse purchases and gifts.

Summary & Key Takeaways

• Thomas Wolf, co-founder and Chief Science Officer of Hugging Face, believes robotics is at the same moment transformers and language models were a handful of years ago. Hardware existed, but the missing brick was adaptive, dynamic software, which recent lab breakthroughs finally began to provide about 18 months ago.

• LeRobot is Hugging Face's effort to reproduce the transformers library's success in robotics, offering a central library that packages the latest algorithms, datasets, and hardware actuators together. The goal is turning the 100 to 200 million software developers into roboticists, just as they became AI-aware, by making tools simple and accessible.

• Hugging Face acquired hardware company Pollen Robotics this year and opened orders for its first robots in mid-July. Wolf wants robots people can vibe code, envisions accessible devices like the $300 Reachy Mini as impulse gifts, and sees entertainment, education, and fun demos as the market where a ChatGPT moment may first arrive.