NVIDIA GTC May 2020 Keynote Pt2: NVIDIA RTX - A New Era for Computer Graphics

TL;DR

NVIDIA RTX advances computer graphics by combining accelerated ray tracing with deep learning to produce high-resolution, photorealistic images efficiently. DLSS 2.0 can transform a 540p or 720p input into an anti-aliased 1080p image using a neural network trained against 16K renders. In Minecraft, this pairing delivers ray-traced lighting, higher resolution, and improved speed. Read on to see how DLSS is trained and why motion vectors matter.

Transcript

Computer graphics is the driving force of NVIDIA. It is one of the world's most computationally intensive applications. It has been for decades and it will continue to be for decades to come. 40 years ago, one of NVIDIA's researchers wrote a seminal paper on a technique to simulate light, we call ray tracing. You trace a light beam through an envir... Read More

Key Insights

• 🙌 NVIDIA's RTX technology combines ray tracing and deep learning to create photorealistic images.
• 🔡 DLSS allows for the synthesis of higher-resolution images from low-resolution inputs, surpassing native rendering quality in some cases.
• 👾 RTX and DLSS greatly enhance the visuals in games like Minecraft, offering real-time ray tracing and high-resolution rendering.
• 💦 NVIDIA Omniverse provides a collaborative platform for designers and artists to work together remotely, leveraging the power of RTX GPUs.

Questions & Answers

Q: What is NVIDIA RTX, and how does it improve computer graphics?

NVIDIA RTX combines accelerated ray tracing with deep learning. Ray tracing models how light reflects, refracts, and illuminates surfaces, while deep learning helps turn relatively low-resolution renders into higher-resolution, anti-aliased images.

Q: What is ray tracing, and how does it simulate light?

Ray tracing follows a light beam through an environment as it bounces off surfaces. By calculating reflection, refraction, and illumination, it can ultimately generate a photorealistic image.

Q: What is DLSS, and how does it generate higher-resolution images?

DLSS stands for deep learning super sampling. It sends a low-resolution image and its motion vectors into an artificial intelligence network, which synthesizes the pixels needed for a higher-resolution image.

Q: How is the DLSS neural network trained?

NVIDIA used a supercomputer to render 16K, anti-aliased images as ground truth. The system compares the neural network's output with those images, propagates the differences back through the network, and adjusts its weights trillions of times.

Q: How does DLSS 2.0 compare with DLSS 1.0 and native 1080p rendering?

DLSS 1.0 improved the apparent resolution of a 720p image only slightly and remained somewhat blurry. DLSS 2.0 generated an anti-aliased 1080p image with more visible detail, and the keynote states that it performed better than native 1080p in the demonstrated scene.

Q: Can DLSS 2.0 upscale a 540p image to 1080p?

Yes. The keynote demonstrates DLSS 2.0 taking a 540p input and recreating it as a 1080p output using a neural network trained on high-quality images generated by a supercomputer.

Q: Why does DLSS use motion vectors?

Motion vectors describe where a pixel is and where it is traveling. By observing pixels across several scenes, the neural network can predict what each scene should look like and recreate details that were not visible in a single low-resolution frame.

Q: How do RTX and DLSS improve Minecraft graphics and performance?

RTX adds real-time ray-traced lighting to Minecraft, where user-created worlds make pre-baked shadows and lighting effects impractical. Ray tracing alone produced only 35 frames per second in the demonstrated scene, while DLSS scaled a low-resolution render to provide ray tracing, high resolution, and higher speed together.

Summary & Key Takeaways

• NVIDIA's researchers developed ray tracing, a technique to simulate light, over 40 years ago.

• NVIDIA RTX, introduced in 2018, combines accelerated ray tracing and deep learning to generate photorealistic images.

• DLSS (Deep Learning Super Sampling) uses artificial intelligence to synthesize higher resolution images from low-resolution inputs.