Notes-while-building-lilLM
January 29, 2025 · 1 min · CohleM
Pytorch Commands I forget time to time/ commands that are essential
January 29, 2025 · 1 min · CohleM
Tokenization
January 22, 2025 · 7 min · CohleM
Papers Summaries
January 21, 2025 · 2 min · CohleM
KV cache and Grouped Query Attention
January 18, 2025 · 11 min · CohleM
RMSNorm
January 15, 2025 · 1 min · CohleM
RoPE
January 15, 2025 · 7 min · CohleM
GPUs
January 8, 2025 · 6 min · CohleM
Mixture of Experts
January 5, 2025 · 14 min · CohleM
DDP and gradient sync
January 3, 2025 · 6 min · CohleM
Gradient Accumulation
January 3, 2025 · 1 min · CohleM
Training Speed Optimization
January 2, 2025 · 3 min · CohleM