How Does NVIDIA’s New Tech Create a Virtual City?

TL;DR

NVIDIA’s new technique creates a virtual city by dividing a large NERF reconstruction into smaller chunks, distributing them across multiple graphics cards, and combining their results. The demonstrated city covers almost 10 square miles and requires up to 64 graphics cards, although its fine details are not very crisp. Read on to understand how NERFs fill gaps between photos, why distributed computing enables larger worlds, and what limitations remain.

Transcript

This paper is insanity. Why? Well, today  you are going to see a revolution in virtual   worlds. You see, we can create a virtual copy of  a real-world scene around us by using NERFs. What   are those? Well, those are an absolutely magical  technique where we take photos of a real scene,   but don’t record the whole thing,  there are holes between ... Read More

Key Insights

• 🕳️ NERF technique synthesizes photos with holes to create virtual scenes.
• 🌍 NVIDIA's solution utilizes distributed algorithms and multiple graphics cards for large-scale virtual world creation.
• 🌍 Future improvements aim to enhance detail quality and reduce the cost of creating virtual worlds.
• 🌍 Human ingenuity drives advancements in virtual world creation technologies.
• 😨 Potential applications include self-driving car simulations and gaming on real tracks.
• 🌍 Challenges such as resource limitations and communication between graphics cards persist in virtual world creation.
• 🌍 Advancements in technology may enable virtual world creation on consumer devices.

Questions & Answers

Q: How does NVIDIA’s new technology create a virtual city?

The technique divides a large virtual-world reconstruction into many smaller problems and distributes those chunks across several graphics cards. Each card handles one part, communicates with the others, and helps assemble the complete city using a distributed algorithm.

Q: What is a NERF, and how does it create a virtual scene?

A NERF takes photos of a real scene that leave holes between the recorded views. It synthesizes what should appear in those missing areas, producing a virtual copy of the scene.

Q: How large is the virtual city created with NVIDIA’s technique?

The demonstrated city spans almost 10 square miles and is 25 times the size of the first example. It was digitized from a collection of photos and turned into a virtual world that can be explored.

Q: What other large environments were demonstrated?

The first example is a racetrack covering one square kilometer, or almost 0.4 square miles. The second is a beach covering 6 square kilometers, almost 2.5 square miles and six times the racetrack’s size.

Q: Why are multiple graphics cards needed?

As a NERF scene grows, a single graphics card runs out of hardware resources. Distributing smaller pieces across several cards provides more total memory, though the cards must communicate to combine their work.

Q: How many graphics cards are required for the virtual city?

Creating the city can require up to 64 graphics cards. The transcript identifies this hardware requirement as one reason the technique is not cheap.

Q: What are the main limitations of NVIDIA’s virtual-city technique?

The fine details are not very crisp, so a super-resolution technique may be needed to estimate them. The process is also expensive because the city can require up to 64 graphics cards.

Q: What could these large virtual worlds be used for?

The transcript suggests training self-driving cars safely in simulation and building racing games on real tracks. It also proposes games involving flight over the reconstructed beach and simulations of what might happen if that beach flooded.

Summary & Key Takeaways

• NERF technique creates virtual scenes by synthesizing photos with holes.

• NVIDIA's solution uses distributed algorithms and multiple graphics cards to create massive virtual worlds.

• The technique has limitations in detail quality and cost but shows immense potential for future advancements.