Google’s New AI Watched 2,500 Videos! But Why?

TL;DR

An amazing AI technique that generates seamless looping videos from photos, using neural stochastic motion texture.

Transcript

This is an incredible AI technique where in  goes a photo, and then, out comes a video of   it. But wait, not any kind of video, even  better, a seamlessly looping video. I love   it. What I found really surprising is that  it works quite well on a variety of cases,   not only on simple periodic movements for flowers,  but for even a sleeping cat, ... Read More

Key Insights

• 🎮 The AI technique can generate seamless looping videos with realistic motions from a single photo.
• 😒 It uses a neural stochastic motion texture to describe how pixels should move when tugged.
• 🐢 The technique can manipulate existing videos by increasing or slowing down motion.
• 🎮 The synthesized motions in the videos are often indistinguishable from reality.
• 💁 Distorting the original image too much can result in missing information and distorted backgrounds.
• 📲 There was a similar technique called Video Textures, developed more than 20 years ago, which could create looping videos from non-looping ones.
• ⏮️ The AI technique represents a significant advancement compared to previous animation techniques.

Questions & Answers

Q: How does the AI technique generate videos from photos?

The AI model analyzes videos to understand motion trajectories and creates a neural stochastic motion texture that describes how pixels should move. This texture is then applied to photos to generate animated videos.

Q: Can the AI technique manipulate existing videos?

Yes, it can magnify the motion in existing videos or create slow-motion versions. The technique uses the same neural stochastic motion texture to modify the movement in the videos.

Q: How realistic are the animated videos generated by the AI technique?

The videos are incredibly realistic, with synthesized motions that are often indistinguishable from reality. It would be challenging to tell that these videos are not real.

Q: Are there any limitations to the AI technique?

One limitation is that excessively distorting the original image can result in missing information and distorted backgrounds. Image inpainting techniques can address the former, but the latter is more challenging to fix.

Summary & Key Takeaways

• AI technique creates seamless looping videos from photos, even for complex scenes like sleeping cats.

• Uses neural stochastic motion texture to describe how pixels should move when tugged.

• Can manipulate existing videos, increasing motion or creating slow-motion versions.