Elon Musk on Tesla FSD v11: One stack to rule them all | Lex Fridman Podcast Clips

TL;DR

Tesla is working on fundamental rewrites of the neural net architecture and software improvements, aiming for version 11 with significant capabilities, but it may take time to iron out any initial issues.

Transcript

so if you look uh 10 point fsd 10.6 just came out recently 10.7 is on the way maybe 11 is on the way somewhere in the future yeah um we were hoping to get 11 out this year but it's uh 11 actually has a whole bunch of uh fundamental rewrites on the neural neural net architecture um and some fundamental improvements in creating vector space uh so um ... Read More

Key Insights

• 💦 Tesla is continuously working on improving its neural net architecture and software, with version 11 expected to bring significant enhancements.
• ✋ The evolution towards neural net-based software is aimed at achieving better capabilities and higher efficiency in the FSD system.
• 🍓 The transition to surround video and raw photon counts will simplify and optimize the training process for Tesla's neural net system.

Questions & Answers

Q: What improvements will version 11 of Tesla's FSD software bring?

Version 11 will introduce fundamental rewrites in the neural net architecture, enabling more capabilities and a single stack solution for all functions. It will eliminate the need for assembling a bag of data points, allowing the neural net to output proper vectors directly.

Q: Why is Tesla shifting towards more neural net-based software?

Tesla believes that over time, neural nets will dominate software as they offer superior capabilities compared to conventional heuristics-based approaches. The move towards more neural net-based software will enhance the performance and efficiency of the FSD system.

Q: What changes will be made in terms of training the neural net system?

Tesla plans to move all training data to surround video and use raw photon counts instead of processed images. This shift will require the redoing of training to train against raw photon counts, reducing the complexity and streamlining the training process.

Q: Will other sensors be incorporated into Tesla's FSD system?

Yes, Tesla aims to incorporate sound or audio into the FSD system to listen for important cues like emergency sirens, vocal communications, or other auditory signals that are crucial for safe autonomous driving.

Summary & Key Takeaways

• Tesla recently released version 10.6 of its Full Self-Driving (FSD) software, with version 10.7 and potentially version 11 in the works.

• Version 11 will include fundamental rewrites of the neural net architecture, allowing for greater capability and a single stack for all functions.

• The current neural net system delivers a bag of data points to the control code, but Tesla aims to refine this process by using additional layers of neural nets to distill the data into vector spaces.