Algorithmic Beautification of Selfies | Two Minute Papers #159

TL;DR

Algorithmic beautification corrects selfie distortions by reconstructing the camera perspective and a 3D face model, then editing the perceived camera distance. The method uses facial hotspots such as the chin, eyebrows, nose stem, eyes, and lips, processes the image in less than 5 seconds, and can also rotate the reconstructed face. Read on to see how the method works, how it is evaluated, and what else it enables.

Transcript

Dear Fellow Scholars, this is Two Minute Papers with Károly Zsolnai-Fehér. Today we're going to talk about a rigorous scientific topic, none other than the creation of the perfect selfie photo. By definition selfies are made by us, which means that these are typically short-range photos, and due to the perspective distortion of the camera lens, we ... Read More

Key Insights

• 📷 Selfie photos often have perspective distortions due to short camera distances.
• 📷 The algorithmic beautification technique involves analyzing the photo, creating a digital model of the perspective camera, and adjusting the camera distance to correct distortions.
• 😀 A 3D model of the face allows for additional enhancements like rotation.
• 😀 Well-recognizable hotspots on the face are used to create an accurate 3D representation.
• 🥡 The technique can be evaluated by comparing its output with ground truth images taken at different camera distances.
• ❓ The technique has been made available as an interactive version online.
• 🤗 The research opens up possibilities for automated techniques to create the perfect selfie photo.

Questions & Answers

Q: How does algorithmic beautification correct selfie photos?

The technique analyzes a selfie to estimate how far away the camera was when the photo was taken. It then models the perspective camera and the face in 3D, allowing the perceived camera distance to be adjusted to even out lens distortions.

Q: Why can close-range selfies make the nose and forehead look larger?

Selfies are typically taken at short range. At that distance, perspective distortion from the camera lens can heavily magnify features such as the nose and forehead.

Q: How does the technique estimate the original camera setup?

It works backward from the input photo using the known optics of perspective cameras. Through a mathematical fitting process, it finds an appropriate camera setup that could have produced the photo.

Q: How is the 3D model of a face created from a selfie?

The method locates recognizable facial hotspots, including the chin, eyebrows, nose stem, region under the nose, eyes, and lips. Because those landmarks alone produce a poor 3D representation, the authors add more hotspots to the detection process.

Q: How long does the selfie-processing technique take?

The expanded hotspot detection and modeling process takes less than 5 seconds. This includes using additional facial hotspots to improve the 3D representation.

Q: Can the reconstructed face be rotated as well as corrected for distance?

Yes. Once the technique has created a digital 3D model, it can rotate the face in multiple directions in addition to changing the perceived camera distance.

Q: How did the researchers evaluate the corrected selfies?

They compared the algorithm's output with real images taken from closer or farther away, treating those photographs as ground truth. The differences were often barely perceptible, and difference images were used to show where errors occurred.

Q: Can anyone try the algorithmic selfie technique online?

Yes. The authors uploaded an interactive version of their work online that anyone can try free of charge.

Summary & Key Takeaways

• Selfie photos often have perspective distortions, such as magnification of the nose and forehead, due to the short camera distance.

• The technique described in the video analyzes the photo, creates a digital model of the perspective camera, and adjusts the camera distance to correct distortions.

• By creating a 3D model of the face, additional enhancements like rotation can be applied to improve the overall selfie photo.