AI Frontiers: Jesper Hvirring Henriksen (OpenAI DevDay)

TL;DR

Be My Eyes introduces Be My AI, a feature powered by GPT-4 Vision, providing blind and low-vision individuals with 24/7 AI assistance for image descriptions and visual support.

Transcript

[music] -Hi, everyone. I'm Jesper and I'm with Be My Eyes. Be My Eyes, we launched a brand new feature, Be My AI on GPT-4V an enclosed partnership with OpenAI. Be My Eyes was founded on the idea that giving blind and low-vision people access to visual assistant through a community of volunteers, through an app. Today we have over half a million bli... Read More

Key Insights

• 👥 Be My Eyes is a platform that connects blind and low-vision individuals with volunteers who can assist them through video calls, but they wanted to provide users with more independence and options.
• 🌐 Be My AI is a new feature developed in partnership with OpenAI, using GPT-4 Vision, which allows blind and low-vision users to receive image descriptions from an AI assistant available 24/7.
• 📱 The AI assistant can describe images encountered online or in apps, making media and photos accessible to those who cannot see them, improving user experiences and inclusivity.
• 🌍 GPT-4 Vision has a wide range of real-world applications, from utility-based functions to helping with navigation, describing humans, pets, landscapes, and buildings.
• 📷 Lack of meaningful alt text in images on websites and apps presents a challenge for blind and low-vision users, but Be My AI can provide thorough descriptions, enhancing accessibility.
• 👵 One user, Caroline, who used to make only two calls a year, has made over 700 image description requests since becoming a Be My AI beta tester in March.
• 🚀 Be My AI has also been integrated into enterprise customer support products, such as Microsoft's Disability Answer Desks, where users can start with a chatbot and rarely need to escalate to a call.
• 🌟 The implementation of AI models that can see, hear, understand, and speak in human-like voices has the potential to greatly improve accessibility in assistive technologies and enhance user satisfaction.

Questions & Answers

Q: How does Be My AI improve accessibility for blind and low-vision individuals?

Be My AI utilizes GPT-4 Vision technology to provide detailed image descriptions and support in navigating the physical and digital world, significantly enhancing accessibility for blind and low-vision users. The AI assistant can assist with real-world tasks such as describing images, reading texts, and providing visual guidance.

Q: What sets GPT-4 Vision apart from previous AI technologies?

GPT-4 Vision stands out from previous AI technologies as it not only accurately describes images but also provides human-like responses and a touch of wit. It has been trained to understand and respond to a diverse range of images, making it highly useful for blind and low-vision individuals in various visual situations.

Q: How has Be My AI been integrated into other platforms besides the Be My Eyes app?

Be My AI has been successfully integrated into Microsoft's Disability Answer Desks, where users can choose to start with a chatbot instead of a call. This integration has led to a significant decrease in call escalations, with 9 out of 10 users finding their issues resolved through the chatbot.

Q: What has been the user response to Be My AI?

The user response to Be My AI has been overwhelmingly positive, with high satisfaction ratings of around 95%. The feedback received indicates that the introduction of AI assistance has been life-changing for blind and low-vision individuals, providing them with increased independence and support in their daily lives.

Summary & Key Takeaways

• Be My Eyes launches Be My AI, an AI-powered feature that offers visual assistance to blind and low-vision individuals.

• The GPT-4 Vision technology allows for thorough image descriptions and support in various real-world and digital use cases.

• Early feedback from users has been overwhelmingly positive, with significant improvements seen in accessibility and user satisfaction.