This AI Creates Human Faces From Your Sketches!

TL;DR

NVIDIA has developed an algorithm that can turn crude sketches into nearly photorealistic images, including human faces, offering a high level of controllability and a variety of possibilities.

Transcript

Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér. In 2017, so more than 300 episodes ago, we talked about an algorithm that took a 3D model of a complex object, and would give us an easy to follow, step by step breakdown on how to draw it. Automated drawing tutorials, if you will! This was a handcrafted algorithm that u... Read More

Key Insights

• 🍳 Automated drawing tutorials have evolved from breaking down 3D models to generating photorealistic images from crude sketches.
• 😀 NVIDIA's algorithm is capable of generating nearly photorealistic landscapes and human faces from simple doodles.
• 🥰 The algorithm allows for customization of hair shape, art-direction through sliders, and even face copy-pasting.
• 😃 While the algorithm has shown great potential, it still has room for improvement, such as matching eye colors and addressing asymmetry in the output.
• 🧑‍🎨 The objective of these techniques is to assist artists in bringing their vision to life by reducing the execution time.

Questions & Answers

Q: How does NVIDIA's algorithm transform crude sketches into photorealistic images?

NVIDIA's algorithm uses a labeling system, where the sketch serves as a guide image. The algorithm then generates a photorealistic image that adheres to the labels, filling in details and refining the output.

Q: What makes the algorithm suitable for generating human faces?

Unlike previous versions, this technique can synthesize human faces, allowing for the creation of realistic facial features. The algorithm also provides options to customize and modify hair shape for the generated images.

Q: What are the additional features of the algorithm?

Apart from refining the output based on changes in the sketch, the algorithm offers controllability through sliders, allowing users to further customize and art-direct the generated images. It also supports a face copy-paste feature, enabling the combination of different aspects of human faces without sketching.

Q: What were the main findings from the user study?

The user study revealed that journeymen users appreciated the variety the algorithm offers, while professionals were more excited about the controllability aspect. Both groups found the algorithm useful in saving time and bringing artistic visions to life.

Summary & Key Takeaways

• In the past, algorithms focused on breaking down complex 3D models into easier-to-draw steps, but now, the focus has shifted to turning crude sketches into photorealistic images.

• NVIDIA's algorithm takes a crude sketch as input and produces a highly detailed and realistic human face that matches the sketch. It allows for customization of hair shape and further art-direction with sliders.

• The algorithm also offers a "face copy-paste" feature, where specific aspects of different human faces can be combined without the need for drawing.