This AI Creates A 3D Model of You! 🚶‍♀️

TL;DR

Technology now allows for the reconstruction of 3D human poses and geometry from still images and videos.

Transcript

Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér. Today, a variety of techniques exist that can take an image that contains humans, and perform pose estimation on it. This gives us these interesting skeletons that show us the current posture of the subjects shown in these images. Having this skeleton opens up the possib... Read More

Key Insights

• 🎮 Pose estimation technology has advanced from still images to video reconstructions.
• 😒 The new method uses image to image translation and inference techniques to reconstruct unobserved data.
• 👻 Higher resolution outputs allow for more detail in the reconstructions.
• ❓ Advancements in learning algorithms have contributed to the progress in pose estimation and reconstruction.
• 🍂 Reconstructed 3D models can be used for various applications such as fall detection, analyzing athletic performance, and activity recognition.
• 👶 The new method shows promising results, but further improvements are expected in future iterations.
• 🎮 The consistency of the reconstruction method enables its extension to video reconstructions.

Questions & Answers

Q: How does pose estimation technology work in still images?

Pose estimation technology uses algorithms to analyze the posture and position of individuals in still images, creating a skeleton-like representation of their poses.

Q: How is it possible to reconstruct the backside of a person when the data is unobserved?

The new method uses image to image translation techniques to estimate the unobserved data. By inferring details based on prior knowledge, such as the shape of clothing or objects, the algorithm reconstructs the backside of the person.

Q: Can this method be applied to video reconstruction?

Yes, while video reconstruction currently has some flickering issues, preliminary results are encouraging. The consistency of the reconstruction method allows for the extension of this technique to videos.

Q: How does the new method compare to previous techniques?

Previous techniques from 2019 lacked detail, making it difficult to recognize the target subject from the reconstructions. The new method, introduced just a year and a half later, shows significant progress and produces more accurate reconstructions.

Summary & Key Takeaways

• Various techniques exist for pose estimation in still images, which can be used for applications such as fall detection and activity recognition.

• With advancements in learning algorithms, it is now possible to reconstruct not only the pose but also the 3D geometry of human models, including body shape, face, and clothes.

• Experiments have shown that this new method can reconstruct both the front and backside of a person, even when the backside is unobserved.