How to detect a sleepy driver | Matt Walker and Lex Fridman

TL;DR

A sleepy driver can be detected by combining computer-vision signals from the eyes with steering and pedal data. Useful signs include blinking, eye movement, partial or full eyelid closures, closure duration, pupil-size changes, steering-wheel reversals, and repeated corrections. Because drowsiness varies by person, algorithms could compare current behavior with each driver’s well-rested driving signature and sleep history. Read on to see how this tailored system could work.

Transcript

i've for for uh five six years at mit really focused on this human side of driving question and one of the big concerns is the uh micro sleeps drowsiness these kinds of ideas and one of the open questions was is it possible through computer vision to detect or any kind of sensors the nice thing about computer vision is you don't have to have direct... Read More

Key Insights

• 🕵️ Computer vision can be utilized to detect drowsiness and microsleeps in drivers without direct contact.
• 😃 Signals such as eye movement, blinking, partial eye closures, eyelid duration, and changes in pupil size can indicate sleepiness.
• ⚾ Combining vision-based signals with steering angle, maneuver, and pedal pressure data can enhance the accuracy of drowsiness detection.
• 😪 Individualized approaches that consider sleep history and behavior patterns may lead to more effective detection systems.
• 🕵️ Detecting drowsiness is crucial as it can prevent accidents caused by drivers falling asleep at the wheel.
• 👨‍🔬 Semi-autonomous vehicles like Tesla Autopilot require further research on how drowsiness affects supervising drivers.
• 👤 The urgency and risk associated with driving might contribute to better vigilance in Tesla Autopilot users, despite the automation.
• 🪛 The addition of a driver-facing camera for drowsiness detection could improve the interaction between humans and machines in autonomous driving scenarios.

Questions & Answers

Q: How can a sleepy driver be detected?

Computer vision can monitor blinking, eye movement, eyelid aperture, partial or full closures, closure duration, and pupil-size changes. Combining those observations with steering angle, steering maneuvers, and pedal pressure could identify when a driver is starting to fall asleep.

Q: Can computer vision detect microsleeps without touching the driver?

Computer vision is attractive because it does not require direct contact with the driver. Video of the face may reveal changes in blinking, eye movement, eyelid closure, and pupil size associated with increasing drowsiness or microsleeps.

Q: Which eye signals may indicate driver drowsiness?

Potential signals include blinking, eye movement, partial eye closure, full eye closure, and how long those closures last. Changes in pupil size may also provide information because the autonomic nervous system helps control pupillary size as a person transitions between states.

Q: What vehicle-control data can strengthen drowsiness detection?

Steering angle, steering maneuvers, and pressure on the pedals can complement camera-based observations. The discussion suggests that a combination of these features could create a recognizable pattern showing that the driver is beginning to fall asleep.

Q: How do some current cars estimate that a driver is sleepy?

Some cars use steering-wheel reversals as their primary signal and may display a coffee-cup warning. Constant corrections can suggest that the vehicle is drifting while the driver enters a microsleep, although the transcript describes this method as crude.

Q: Why might one drowsiness signal not work for every driver?

Behavior can change in different ways as individual drivers become sleepy, so a single signal may not be reliable for everyone. Even within a set of 10 possible features, one person might show seven cardinal features while another shows six that only partly overlap.

Q: How could personalized driver-drowsiness detection work?

An algorithm could first learn a person’s driving behavior when they are well rested. It could then compare current behavior with that baseline and the individual’s sleep history, using deviations to build a tailored set of sleepiness indicators.

Q: Why can falling asleep at the wheel be especially dangerous?

With drugs or alcohol, a driver may react too late, but a sleeping driver does not react at all. The transcript describes the resulting vehicle as a two-ton missile moving down the street with no one in control.

Summary & Key Takeaways

• Researchers have focused on using computer vision to detect drowsiness and microsleeps in drivers without direct contact.

• Possible signals of sleepiness include eye movement, blinking, partial closures, eyelid duration, and changes in pupil size.

• Combining these signals with aspects of steering angle, steering maneuver, and pedal pressure could create a reliable detection system.