GTC Japan 2017 Part 6: New NVIDIA TensorRT 3

TL;DR

NVIDIA's TensorRT and Volta GPUs offer significant speed and efficiency improvements for deep neural networks, enabling faster image recognition and natural language translation.

Transcript

so we the tensor RT takes this computational graph and runs it through an optimizing compiler and outputs a Volta CUDA program this Volta could a program was written by software so software wrote software and this Volta could a program for a deep neural network could predict many things depends on what you taught it to do the performance that we've... Read More

Key Insights

• 🉐 TensorRT, combined with Volta GPUs, offers exceptional performance gains for deep neural networks in both image recognition and natural language translation.
• ⌛ The speed improvements achieved by TensorRT and Volta GPUs compared to CPUs are staggering, with gains of thousands of times in image recognition and hundreds of times in natural language translation.
• 👻 Volta GPUs drastically reduce the infrastructure required for image recognition, allowing for significant cost and energy savings.

Questions & Answers

Q: How does TensorRT improve the performance of deep neural networks?

TensorRT takes a computational graph and compiles it into an optimized Volta CUDA program, resulting in significantly faster performance compared to traditional CPUs.

Q: What is the speed difference between running image recognition on CPUs versus V100 GPUs?

With TensorRT and V100 GPUs, image recognition speed is a remarkable 5,740 times faster than using CPUs, allowing for real-time processing of large volumes of images.

Q: How does TensorRT improve natural language translation speed?

TensorRT enables a 100-fold speed increase in natural language translation, allowing for faster translation from English to Chinese, German, French, and other languages.

Q: How does Volta GPUs revolutionize image recognition infrastructure?

Volta GPUs can achieve image recognition at a rate of 45,000 images per second, requiring only one server with 8 GPUs, which is a significant improvement in efficiency and cost compared to traditional CPU-based solutions.

Summary & Key Takeaways

• TensorRT optimizes computational graphs and generates Volta CUDA programs, providing amazing performance gains for deep neural networks.

• Using TensorRT with TensorFlow, image recognition speed on V100 GPUs is 5,740 times faster than CPUs, while natural language translation is 100 times faster.

• Volta GPUs can achieve image recognition at a rate of 45,000 images per second, requiring only one server with 8 GPUs and consuming 3000 watts.