Stanford CS336 Language Modeling from Scratch | Spring 2025 | Lecture 13: Data 1

Transcript

So today's lecture is going to be on data. In the previous lectures, up until now, we've discussed how you train a model, given data. So we've talked about the architecture. We've talked about the optimizer, tokenization, scaling laws, parallelism. That's all given a fixed data set. And now, we're going to talk about what data do we train on. So my... Read More