Stanford CS336 Language Modeling from Scratch | Spring 2025 | Lecture 6: Kernels, Triton

Transcript

Today, we're going to be going into details on writing high performance code for GPUs. So part of assignment 2 is going to be you're going to have to do a bunch of profiling, you will have to write your own Triton kernel for FlashAttention-2. You will need to make all of this stuff very high performance. And so in this lecture, we're going to drill... Read More