🌁#76: Rethinking Scaling Laws (when plateau is actually a fork)
we discuss Compound AI and test-time compute to challenge the scaling plateau and pave the way for smarter AI systems, along with our usual collection of interesting articles, relevant news, and research papers. Dive in!
Mixture-of-Depths lets transformers skip layers for low-priority tokens, cutting FLOPs by up to 50%. Learn how MoD routing works and how it compares to MoE.
Metacognitive AI — LLMs that monitor their own limits — is the next research frontier. Covers AI welfare, anthropomorphism debates, and top papers of the week.
FOD#74: Sparks of AGI – OpenAI’s plans to get there
we discuss Sébastien Bubeck at OpenAI, what is going on with SLMs, and offer you a collection of interesting articles, relevant news and research papers. Dive in!
We discuss Apple’s unfortunate underdelivery and provide a collection of immensely interesting articles, interviews, news, and research papers. Dive in!