Unveiling game-changing wearable tech | Pattie Maes

TL;DR

The wearable system acts as a portable “sixth sense,” projecting relevant digital information onto any surface and letting users interact through natural gestures. Built from a webcam, battery-powered projector, mirror, and cell phone, the prototype cost $350 using off-the-shelf components. It can recognize products, annotate books and newspapers, display travel updates, and capture photos. Read on to see how these capabilities work.

Transcript

I've been intrigued by this question of whether we could evolve or develop a sixth sense -- a sense that would give us seamless access and easy access to meta-information or information that may exist somewhere that may be relevant to help us make the right decision about whatever it is that we're coming across. And some of you may argue, "Well, do... Read More

Key Insights

• 🤔 We lack a seamless and easy way to access relevant information that can help us make optimal decisions in various situations.
• 💡 A research group at the Media Lab has developed a wearable device that provides access to relevant information through natural gestures.
• 💻 The device consists of a camera, a portable projection system, and a cell phone acting as a communication and computation device.
• 🌍 The system allows users to interact with information projected onto any surface, including their hand, using gestures and movements.
• 🛍️ In a supermarket, the device can recognize products and provide information like eco-friendliness or personal preferences.
• 📚 When interacting with a book, the device can display reader comments, expert annotations, and Amazon ratings.
• 🗞️ Reading the newspaper becomes interactive with video annotations and real-time updates on events and sports scores.
• 👥 The device can provide word clouds of tags associated with people you meet, giving insights into their interests and background.

Questions & Answers

Q: How does the wearable device track gestures and interact with projected information?

A camera tracks four significant fingers, which are marked with colored caps or nail paint, and recognizes the user’s gestures. The user can then manipulate projected information with natural hand movements, such as zooming on a map, resizing pictures, or organizing them.

Q: What components make up the wearable system?

The prototype combines a simple webcam, a portable battery-powered projection system with a small mirror, and a cell phone. The phone, carried in a pocket, serves as the system’s communication and computation device.

Q: How much did the wearable prototype cost?

The prototype cost $350 using components available off the shelf at that time. Pattie Maes also says a mass-produced version would not cost more than the cell phones of that period.

Q: Can the device project information onto any surface?

Yes, the portable system lets the user approach and project information onto any available surface. It can even use the wearer’s hand when another surface is unavailable.

Q: How does the device help users evaluate products while shopping?

The system recognizes a selected product through image recognition or marker technology. It can display a green or orange light and provide additional information based on the user’s personal criteria, such as a preference for an ecologically responsible choice.

Q: What information can the device display about books?

The device can project an Amazon rating directly onto a book’s cover. As the user turns pages, it can also show reader comments, information from a preferred critic, and annotations from an expert or friend.

Q: Can the wearable device provide current information and travel updates?

Yes, it can add video annotations and the latest sports scores to newspaper stories. When the user picks up a boarding pass, it can also report that a flight is delayed or its gate has changed.

Q: What other natural gestures does the system recognize?

The system recognizes an iconic “take a picture” gesture and photographs whatever is in front of the wearer. It also lets the user find the current time by drawing a watch on an arm.

Summary

This talk discusses the idea of developing a sixth sense that would provide us with seamless access to relevant information in order to make optimal decisions. The speaker introduces a wearable device that allows users to interact with projected information using natural gestures. The device is portable and can be used on any surface. It can recognize products, provide ratings and additional information, annotate books, display video annotations in newspapers, show word clouds associated with individuals, and even provide real-time flight updates. While the device is still a work in progress, it has the potential to revolutionize the way we access and utilize information.

Questions & Answers

Q: How does the speaker describe the current availability of relevant information?

The speaker argues that we don't have easy access to relevant information that can help us make optimal decisions in various situations. Despite the existence of cell phones, we don't usually take them out and start searching for information during face-to-face interactions or when making purchasing decisions in a supermarket.

Q: What is the speaker's research group at the Media Lab working on?

The speaker's research group has been developing inventions to provide easy access to relevant information without requiring users to change their behavior. They have created a wearable device consisting of a webcam, projection system, and a cell phone as the communication and computation device.

Q: How does the device allow users to interact with information?

The device tracks the four significant fingers using marker caps or painted nails. It recognizes gestures made by the user, allowing them to interact with projected information on any surface. For example, users can zoom in and out on a map, take pictures, sort and organize images, and use natural gestures to navigate through data.

Q: What is the difference between this device and other interactive systems?

While other interactive systems like the Microsoft Surface Table also use natural gestures, this device has the advantage of being portable and usable on any surface. Users can walk up to any surface, including their own hand if necessary, to interact with the projected data.

Q: What are some potential applications of the device?

The device can provide real-time information and ratings about products while shopping. It can display ratings and additional information about books, including reader comments and expert critiques. It can even display video annotations and real-time sports scores while reading newspapers. Additionally, it can show word clouds associated with individuals and give real-time flight updates on boarding passes.

Q: What are some limitations of the current version of the device?

The speaker acknowledges that the device is still a work in progress and not yet perfect. While it is a significant achievement, there is room for improvement and further development. The speaker also mentions the possibility of a future ultimate sixth sense brain implant.

Q: Who is the genius behind the development of this device?

The speaker highlights his student, Pranav Mistry, as the genius who has been designing and implementing the system. Pranav's dedication and hard work have been crucial to the device's development.

Q: How does the speaker envision the device's mass production?

The speaker believes that, in mass production, the device would not cost more than today's cell phones and could be even more stylish and compact. The goal is to make the device accessible and appealing to a wide range of users.

Q: How does the speaker summarize the significance of the device?

The speaker emphasizes that this device has the potential to act as a sixth-sense device, providing users with relevant information about whatever is in front of them. By seamlessly accessing this information, users can make better decisions and navigate the world more effectively.

Q: What does the speaker express about the future of this technology?

Although the current device is not perfect, the speaker hints at the possibility of an ultimate sixth sense brain implant in the future. This suggests that the development of this technology is ongoing, and there may be even more advanced iterations to come.

Takeaways

This talk introduces a wearable device that aims to provide users with seamless access to relevant information. By using natural gestures, users can interact with projected data on any surface, enabling them to make optimal decisions in various situations. The device has the potential to revolutionize the way we access and utilize information, making it accessible, mobile, and stylish. While the current version described in the talk is a work in progress, the speaker hints at a future where this technology could be even more advanced.

Summary & Key Takeaways

• The researcher and his team have developed a wearable device that allows users to access relevant information in a seamless and easy way.

• The device uses a camera, projection system, and cell phone to track hand gestures and interact with projected information on any surface.

• Users can use the device to make optimal decisions in various scenarios, such as shopping, reading, and interacting with people, among others.