AI 101
+1
14 min read
Mar 26, 2025
Slim Attention, XAttention, Multi-token Attention & KArAt: Four New Attention Mechanisms Explained
We explore four advanced attention mechanisms which improve how models handle long sequences, cut memory use and make attention learnable
AI 101
+1
11 min read
Mar 12, 2025
How to Reduce Memory Use in Reasoning Models
How LightThinker and Multi-Head Latent Attention (MLA) reduce memory use and speed up inference in reasoning models like DeepSeek-R1.
AI 101
+1
13 min read
Mar 5, 2025
Everything You Need to Know about Knowledge Distillation
This is one of the hottest topics thanks to DeepSeek. Learn with us: the core idea, its types, scaling laws, real-world cases and useful resources to dive deeper
