Facebook's Head Of Augmented reality on its plans for AR glasses

TL;DR

There is a need to balance the hype around augmented reality with actual usable products that create value and solve problems for people.

Transcript

bless Brian's heart he thinks college kids still use Facebook no no I know you challenged you guys are building the coolest youngest hippest part of Facebook I agree we are in this unfortunate era of augmented reality demo itis whether it's magic leap showing slide decks that are never actually gonna be working products or might be a really long ti... Read More

Key Insights

• 🥺 Hype around augmented reality can lead to disillusionment among users when real products do not materialize.
• 😒 Utility-focused use cases for AR are still a challenge due to the limitations of handheld devices.
• 😒 Facebook is focused on inspiring developers with exemplary experiences but allows them to come up with new use cases themselves.
• 👪 Privacy implications and control over in-home AR experiences are significant concerns for the future.

Questions & Answers

Q: Is there an unhealthy element to the current hype around augmented reality?

Yes, there is an unhealthy element to the hype, as many companies showcase tech demos that may never become real products that people can use. It's important to have a balance between inspiring demos and actual usable products.

Q: Why haven't we seen more progress in utility-focused augmented reality experiences?

One reason is that most AR experiences are still done on handheld devices like phones. People are not willing to hold their phones out in front of them all day. Additionally, the activation energy required to use AR experiences on phones can be a barrier.

Q: How does Facebook guide the developer community in AR product development?

Facebook focuses on creating lighthouse or exemplary experiences to inspire developers. They want to show what is possible with the platform while allowing developers to come up with new and creative use cases themselves.

Q: What are some promising use cases of AR that Facebook is exploring?

Facebook is actively exploring AR in commerce and gaming. They are running tests for try-on experiences for sunglasses and makeup. In gaming, they are experimenting with short burst experiences that work well on camera devices.

Summary

In this video, Brian acts as an interviewer and speaks with a representative from Facebook about the current state of augmented reality (AR). They discuss the issue of tech demos that are never turned into usable products and the need for a balance between showcasing exciting new technology and creating practical, usable products. They also talk about the lack of progress in certain areas of AR, such as overlaying information in real-world contexts like menus and directions. The conversation touches upon the challenges of developing for handheld devices versus headsets, the importance of guiding the developer community, and the value of creating exemplary experiences. The interview concludes with a discussion on the future of AR, including opportunities in commerce and gaming, the need for an open standard for AR maps, and Facebook's involvement in building AR headsets.

Questions & Answers

Q: Is there an unhealthy element to the hype around AR tech demos?

Yes, there is an unhealthy element to the hype surrounding AR tech demos. While it is important to inspire people about the future of AR, many tech demos are never turned into real products that can be used by people. It is crucial to strike a balance between showcasing cool tech and actually building products that create value, solve problems, and bring joy to people's lives.

Q: Why haven't we seen much progress in the utility space of AR?

One of the reasons for the lack of progress in the utility space of AR is the reliance on handheld devices like smartphones. While there are interesting things that can be done with handheld devices, there are limitations due to the fact that people are lazy and don't want to hold their hand out in front of them all day long. This poses challenges for use cases like augmented street signs and virtualized menus, as it requires too much activation energy and involves a lot of friction to use these features on a handheld device. There is a need to find use cases that are valuable and promising enough to overcome this activation energy and provide a seamless user experience.

Q: How does Facebook guide the developer community in building AR products?

Facebook aims to provide examples of exemplary experiences to inspire the developer community. While the company believes in leaving some freedom for developers to explore their own use cases, they also want to showcase what is possible with the platform and technology. Facebook has artists, prototypers, and product development teams working closely together to influence the creation tools and offer guidance. However, they understand that it is not possible to fully constrain people's creativity, and they welcome new and creative use cases that go beyond their own examples.

Q: Can you provide an example of an exemplary AR experience built by Facebook?

One of the examples mentioned is Facebook's Portal Storytime. This product allows grandparents to video chat with their grandchildren and have an augmented reality storytelling experience. The text of the children's book is overlaid on the screen, and elements from the story, such as characters or masks, are incorporated into the video chat. This experience enhances the visual aspect of storytelling and provides a more structured experience for little kids, making it more enjoyable for both grandparents and grandchildren. It is an example of using AR in a way that goes beyond self-expression and adds value to people's lives.

Q: Does Facebook focus on self-expression and utility, or are there other areas like commerce and gaming?

Facebook is focused on expanding beyond self-expression to areas like commerce, gaming, and education. They aim to add value to where people already use Facebook products and services. In terms of commerce, Facebook is running tests where users can try on sunglasses or makeup using AR before making a purchase. This helps save time and provides a social layer for sharing different looks with friends. In gaming, Facebook is exploring short-burst experiences that work well on mobile devices and do not require extended periods of holding up the camera. They are still in the early stages of exploring gaming use cases for AR.

Q: Is there an open standard for AR mapping, or are there competing standards?

While it would be ideal to have an open standard for AR mapping, the interviewee believes that we are still trying to figure out the use cases and the level of precision required for these maps. Currently, there are no use cases that demand a unified mapping system, and the focus should be on finding real use cases that motivate people to do the actual mapping. However, it is important to start working on this problem now, especially in the context of headsets, and ultimately arrive at a solution that allows for interoperability rather than having separate, siloed mapping systems.

Q: How does Facebook address the privacy implications of AR, especially when it comes to capturing and digitizing private spaces?

Facebook believes in the principle of people being informed and in control of their data. However, addressing the privacy implications of AR is not an easy task. It involves not just technical challenges but also policy and interface design challenges. They are aware of the importance of getting these aspects right to ensure that people's privacy is respected and that they feel comfortable using AR. It is a critical aspect that needs to be solved for AR to reach its full potential.

Q: Is Facebook working on an AR headset?

While the interviewee does not have a specific product to announce at the moment, he acknowledges that Facebook is doing a lot of research in the field of AR and VR. They have talented researchers working on cutting-edge technology and hope to contribute to the future of headsets. While the ultimate vision is to have AR glasses, they recognize the importance of differentiating between on-the-go experiences and home-based immersive experiences. The convergence of these experiences may happen at the software level, but there will likely be separate devices for different use cases.

Q: Will Facebook integrate its research into its own headset or other hardware products?

The research being conducted by Facebook's Reality Labs is intended to advance the state of the art in AR and VR. Some of this research is already being applied to the company's existing Oculus headsets. While Facebook wants to see AR glasses become a reality, they are also open to collaborating with other hardware manufacturers. The primary goal is to contribute to the development of future headsets and help bring them to market. The focus is on advancing the technology and fostering innovation rather than exclusively building their own hardware products.

Takeaways

AR tech demos should be balanced with the creation of usable products that solve real problems and provide value to people. Finding use cases for AR that work well within the limitations of handheld devices is challenging but necessary for progress. Facebook aims to guide developers by providing lighthouse examples and exemplary experiences. The company believes in setting examples while allowing for creative exploration. There is a need for progression in the utility space of AR, such as overlaying information in real-world contexts. Privacy implications of AR and the need for informed and controlled data are critical issues to address. Facebook is conducting research in the field of AR and VR and hopes to play a part in the development of future headsets. They are open to collaborations and envision a future where different experiences converge at the software level.

Summary & Key Takeaways

• Many companies, including Facebook, are guilty of showcasing cool tech demos that lack a clear path to becoming real, usable products.

• The challenge lies in finding utility-focused use cases for augmented reality, such as overlaid photo menus in restaurants or virtual directions in the real world.

• Facebook is working on guiding the developer community by creating exemplary experiences that inspire and provide a starting point for new AR innovations.