Notes

Backpropagation

• Manual Backpropagation On Tensors
• Backpropagation From Scratch

Loss function

• Maximum Likelihood Estimate As Loss
• Why We Need Regularization

Optimization

• Rmsnorm
• Skip Connections
• Optimization Algorithms
• Diagnostic Tool While Training Nn
• Optimizing Loss
• Batchnormalization

Training

Misc

• Matrix Visualization

Architecture Implementation

• Multi Head Latent Attention
• Lora
• Kv Cache Gqa
• Rope
• Mixture Of Experts
• Gpt Implementation

GPU

• Gpus

Interpretability

• Interpretability