Architectures
The structural blueprints behind AI models: transformers, mixture-of-experts, state-space models, diffusion, and JEPA — and what each design makes possible.
AI 101
+1
10 min read
Feb 4, 2026
AI 101: Conditional Memory and the Rise of Selective Intelligence
Engram by DeepSeek gives LLMs conditional memory: models learn when to access knowledge. Covers architecture, U-shaped allocation law, and reasoning gains.
AI 101
+3
10 min read
Jan 14, 2026
What Are Web World Models? How AI Agents Get Persistent Worlds
A practical guide to Web World Models, the web-based architecture for building persistent, controllable environments for AI agents.
AI 101
+1
11 min read
Jan 8, 2026
DeepSeek mHC: Breaking the Architectural Limits of Deep Learning
DeepSeek's mHC fixes 3000× signal amplification in 27B models by constraining residual streams to the Birkhoff Polytope — with only 6.7% training overhead.
AI 101
+1
13 min read
Dec 3, 2025
Fusing Modalities: Basics + the New MoS Approach
Multimodal fusion is how AI combines text, images, and audio into one model. Covers early, late, intermediate fusion types and Meta's MoS approach.
AI 101
+1
13 min read
Nov 19, 2025
LeJEPA: Provable Self-Supervised Learning Without Heuristics
LeJEPA by Yann LeCun: provably stable self-supervised learning without heuristics. Covers SIGReg, isotropic Gaussian embeddings and JEPA improvements.
AI 101
+3
12 min read
May 28, 2025
Decoding BERT: From Original NLP Game-Changer to Today's Efficient AI (feat. ConstBERT)
What is BERT in NLP? Learn how BERT works—MLM, NSP, fine-tuning—plus modern variants like RoBERTa, DistilBERT, ModernBERT, and ConstBERT in 2026.
AI 101
+2
12 min read
Apr 30, 2025
Can Liquid Models Beat Transformers? Meet Hyena Edge – the Newest Member of the LFM Family
What are Liquid Foundation Models? LFM2.5 architecture, Hyena Edge, open-weight availability, benchmarks, and efficiency vs Transformers.
AI 101
+1
14 min read
Mar 26, 2025
Slim Attention, XAttention, Multi-token Attention & KArAt: Four New Attention Mechanisms Explained
We explore four advanced attention mechanisms which improve how models handle long sequences, cut memory use and make attention learnable
AI 101
+1
6 min read
Feb 19, 2025
What Is Mixture-of-Mamba and How Does It Work?
Mixture-of-Mamba (MoM) brings modality-aware sparsity to Mamba SSM, reaching the same loss on less than half the training FLOPs across text, image, and speech setups.
AI 101
+1
8 min read
Jul 17, 2024
xLSTM Explained: Extended LSTM vs Transformers
xLSTM extends classic LSTM with exponential gating and matrix memory. Learn how it compares to Transformers, what sLSTM and mLSTM are, and when to use it.
AI 101
+1
8 min read
Jul 3, 2024
What is KAN?
KAN (Kolmogorov-Arnold Networks) replaces fixed activation functions with learnable splines. How KAN works, how it compares to MLP, and where it falls short.
AI 101
+1
18 min read
Jun 12, 2024
What Is JEPA? LeCun Architecture & World Models
Joint Embedding Predictive Architecture predicts abstract representations, not pixels or tokens. Covers I-JEPA, V-JEPA, VL-JEPA, LeJEPA and robotics.
