PyTorch vs Tinygrad vs Mojo: Which is better? | George Hotz and Lex Fridman

TL;DR

The speaker discusses importing models into Tiny Grad, the challenges faced, and the benefits of using Onyx export. They also talk about the differences between Tiny Grad and other programming languages like Mojo and GML, as well as Tiny Grad's focus on performance optimization.

Transcript

you've gotten llama into tiny grad you've gotten stable diffusion in Italian grind what was that like can you comment on like what are um what are these models what's interesting about boarding them so what's yeah like what what are the the challenges what is what's naturally what's easy all that kind of stuff there's a really simple way to get the... Read More

Key Insights

• 👻 Exporting models as Onyx and using Tiny Grad allows for cleaner code and improved performance.
• 🧘 Tiny Grad is positioned between Mojo and GML in terms of its approach to model optimization and target hardware.
• 🍉 PyTorch's nn.relu class is a subject of criticism in terms of software engineering.
• 🤗 Tiny Grad is currently competitive in terms of performance on Qualcomm GPUs, being used in the open pilot driving model.

Questions & Answers

Q: What is the process of importing models into Tiny Grad and what are the advantages of exporting them as Onyx?

To import models into Tiny Grad, they can be exported as Onyx and then run in Tiny Grad. Onyx export simplifies the code and makes it easier to read. It also allows Tiny Grad to have a cleaner front end and improve performance.

Q: What are the challenges faced in Tiny Grad compared to PyTorch, particularly in relation to nn.relu?

The speaker discusses a software engineering complaint about PyTorch's nn.relu being a class. They argue that since it is stateless, it should be a function, not a class. However, this aspect does not impact performance.

Q: What are the differences between Mojo and GML, and how do they compare to Tiny Grad?

Mojo and GML are programming languages related to Tiny Grad. Mojo takes a breadth-first approach, aiming to make all of Python fast, while GML focuses on running Llama fast on Mac. Tiny Grad is positioned in the middle, striving to make neural networks fast and optimizing for specific hardware.

Q: What is the speaker's goal regarding accelerator development in Tiny Grad?

The speaker's immediate goal is to build a performance stack comparable to PyTorch on Nvidia and AMD GPUs but with fewer lines of code. Once the framework is established, they plan to work on developing an accelerator.

Summary & Key Takeaways

• The speaker explains that models like Llama, Stable Diffusion, and Whisper were ported to Tiny Grad as academic exercises, resulting in cleaner code and fewer lines.

• They highlight a software engineering complaint about PyTorch's nn.relu being a class, and discuss Tiny Grad's cleaner and stateless front end.

• The speaker mentions Mojo and GML, two programming languages related to Tiny Grad, and explains their different approaches and positioning in the spectrum of model optimization.