This AI Learned How To Generate Human Appearance

TL;DR

Neural networks can generate realistic images of human bodies in different poses, even from low-quality input images.

Transcript

Dear Fellow Scholars, this is Two Minute Papers with Károly Zsolnai-Fehér. In this series, we often discuss that neural networks are extraordinarily useful for classification tasks. This means that if we give them an image, they can tell us what's on it, which is great for self-driving cars, image search, and a variety of other applications. Howeve... Read More

Key Insights

• ❓ Neural networks can be used for image generation tasks, not just classification.
• ❓ Generating full human bodies is challenging due to dataset variation.
• 🎮 The technique in the video can synthesize both shape and appearance of human bodies.
• 🫵 It can create views from new angles that were not in the original dataset.
• 🔠 The technique behaves similarly for different inputs, a desirable property.
• 📷 Appearance transfer can be achieved by providing a photo of a different object.
• 🎮 The technique requires artistic control to determine the desired appearance.

Questions & Answers

Q: How do neural networks generate high-resolution images of imaginary celebrities?

By using a generative adversarial network, where two neural networks compete against each other, realistic images can be created.

Q: Which type of images are neural networks good at generating?

Neural networks are great at generating images of faces but struggle with synthesizing full human bodies due to too much variation in the datasets.

Q: How does the technique in the video transform a test subject into a different pose?

The algorithm runs pose estimation on the input image and uses that information to transform the test subject into the desired pose, even creating new angles.

Q: Can the technique generate realistic images from crude drawings?

Yes, the algorithm supports a feature where a crude drawing can be provided, and it will transform it into a photorealistic image.

Summary & Key Takeaways

• Neural networks can be used for image generation, not just classification tasks.

• A new technique can synthesize entire human bodies, both shape and appearance, from minimal input.

• The technique can also create views from new angles and perform appearance transfer based on different input images.