Seyed Masoud Hosseini · Overview · Study log · Ideas · Transcript · RSS feed

Language Modeling from Scratch

Stanford CS336 · AI · Fall 2029 · Planned

Lectures

  1. Lecture 1: Overview and Tokenization (1:18:59)
  2. Lecture 2: PyTorch and Resource Accounting (1:19:22)
  3. Lecture 3: Architectures and Hyperparameters (1:27:03)
  4. Lecture 4: Mixture of Experts (1:22:04)
  5. Lecture 5: GPUs (1:14:21)
  6. Lecture 6: Kernels, Triton (1:20:22)
  7. Lecture 7: Parallelism 1 (1:24:42)
  8. Lecture 8: Parallelism 2 (1:15:10)
  9. Lecture 9: Scaling laws 1 (1:05:18)
  10. Lecture 10: Inference (1:22:52)
  11. Lecture 11: Scaling laws 2 (1:18:13)
  12. Lecture 12: Evaluation (1:20:48)
  13. Lecture 13: Data (1:19:06)
  14. Lecture 14: Data Filtering and Deduplication (1:19:11)
  15. Lecture 15: Alignment - SFT and RLHF (1:14:51)
  16. Lecture 16: Alignment - RL 1 (1:20:32)
  17. Lecture 17: Alignment - RL 2 (1:16:08)

Notes

No notes yet.

References

No references yet.

Study log

No log entries for this course yet.