This Neural Network Learned The Style of Famous Illustrators

TL;DR

GANILLA is a new technique that successfully preserves content and transfers style in image style transfer, thanks to the usage of skip connections.

Transcript

Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér. In the last few years, we have seen a bunch of new AI-based techniques that were specialized in generating new and novel images. This is mainly done through learning-based techniques, typically a Generative Adversarial Network, a GAN in short, which is an architecture wh... Read More

Key Insights

• ⚾ AI-based techniques, like CycleGAN and DualGAN, have improved image style transfer in recent years.
• 🖤 CycleGAN excels in style transfer but sacrifices content preservation, while DualGAN focuses on content preservation but lacks strong style transfer.
• 😒 GANILLA combines the advantages of both techniques with the use of skip connections, achieving impressive results in preserving content and transferring style.
• 👤 User studies have favored GANILLA over previous techniques.
• ❓ GANILLA can accurately reproduce distinct artistic styles, including the style of Hayao Miyazaki.

Questions & Answers

Q: How do GANs work in image style transfer?

GANs consist of a generator network that creates new images and a discriminator network that learns to distinguish real photos from generated ones. The two networks improve together over time, generating better images.

Q: What is the main drawback of using CycleGAN for style transfer?

CycleGAN is heavy-handed and does not preserve the content of the image while transferring style, resulting in a complete transformation of the image.

Q: How does DualGAN address the limitations of CycleGAN?

DualGAN uses two GANs to perform image translation, preserving the content of the image. However, it sometimes adds too little of the style, making it less prominent in the output images.

Q: What makes GANILLA different from previous techniques?

GANILLA successfully preserves content and transfers style simultaneously by incorporating skip connections, which help retain content information as the neural network goes deeper.

Summary & Key Takeaways

• AI-based techniques, such as CycleGAN and DualGAN, have been developed for image style transfer.

• CycleGAN excels in transferring style but compromises the content, while DualGAN preserves the content but lacks in style transfer.

• GANILLA, a new technique, successfully combines content preservation and style transfer, thanks to the use of skip connections.