Do you feel this shift too? The idea of models learning endlessly is showing up everywhere. We see it, we hear it, and it’s all pushing the spotlight toward continual learning.

This approach refers to the ability to keep learning new things over time without forgetting what you already know. Humans do this naturally (as Ilya Sutskever also noted) and they are very flexible to changing data. But, unfortunately, neural networks are not. When developers change the training data, they often face something that is called catastrophic forgetting: the model starts loosing its previous knowledge, and returns to training model from scratch.

Finding the very balance between a model’s plasticity and its stability in previously learned knowledge and skills is becoming a serious challenge right now. This paradigm is the path to more “intelligent” systems that will save time, resources, and money spent on training, it helps mitigate biases and errors, and, in the end, things can just go easier and more naturally with model deployment.

Today we’ll look at the basics of continual learning and two approaches that are worth your attention: very recent Google’s Nested Learning and Meta FAIR’s Sparse Memory Finetuning. There is a lot to explore →

In today’s episode, we will cover:

Continual learning means learning step-by-step from data that changes over time. So it is related to two main things:

The new pieces of information can be new skills, new examples, new environments, or new contexts. As the data comes in gradually, continual learning is also known as lifelong learning. The process of continual learning happens when the model is already deployed.

Everything would be great if models didn’t face one major challenge – catastrophic forgetting. This problem generally looks like this: a neural network is trained on Task 2 after Task 1, and its weights are updated for Task 2. This often pushes them away from the optimum for Task 1, and the model suddenly performs very poorly on that task.

The problem here is not the model’s capacity – this usually happens because of the sequential training procedure. Even in 1989-1990, Michael McCloskey and Neal J. Cohen and R. Ratcliff identified this problem and showed that simple networks lose previous knowledge extremely quickly when trained sequentially. They also highlighted that this forgetting is much worse than in humans.

But if you train on Tasks 1 and 2 interleaved, forgetting does not happen.

Preventing forgetting is only one part of the solution. Effective lifelong learning also requires:

If tasks are related, the model should get better at one after learning another, which marks positive knowledge transfer:

So, a good continual learning system needs the right balance: it should stay stable (not forget old things) while still being plastic enough to learn new ones. It also needs to handle differences within each task and across different tasks. How is it released on practice?

Continual learning is mainly about moving from one task to the next while keeping performance stable or improving it during ongoing learning. That’s why two fundamental setups are used for it:

Continual learning researchers often uses three main scenarios to describe what the model is expected to know at test time and whether it gets task identity information. Importantly, these scenarios are defined by how the changing data relates to the function the network must learn:

These three scenarios are independent of whether the learning setup is task-based or task-free. Here are also some helping methods that can make continual learning even more effective.

Techniques usually refer to methods that improve the main approach or concept. Here are some techniques that work for continual learning and help mitigate the forgetting problem:

These are the general techniques, the basics, that pop up here and there across the field. But right now we’re seeing breakthroughs that give continual learning a fresh perspective and bring it back into focus. And we couldn’t ignore the two most interesting ones →

The first one comes from Google Research. The team noticed that the brain can learn continuously because different parts of it update at different speeds: some areas change quickly, like short-term memory, while others update slowly, like long-term memory. This insight became the basis for their new method, Nested Learning (NL), which brings this idea into machine learning by letting different parts of a model update on different time scales.

The Google researchers noted that if you look at models like Transformers through this lens, they’re basically layers that update at different “frequencies”, similar to what the brain does.

NL is build to overcome the main problem of LLMs that usually remember the context window (short-term memory), remember what was in their parameters from pretraining stage (long-term memory), but can’t store new long-term memories and can’t learn new abilities. This highlights the main gap: new information in context never updates long-term memory and so doesn’t change long-term parameters. So →

NL encourages us to rethink neural networks from the perspective of memory. Instead of seeing a model as one monolithic system trained with one optimizer, NL views it as a stack of smaller learning systems, each operating at its own speed. Each part of the model trains at its own pace and learns from its own “slice” of information, and NL shows that together they form a collection of interconnected optimization problems. But how?

Every part of a modern neural network – layers, optimizers, even attention – can be interpreted as a kind of associative memory with its own update rule.

An associative memory is the ability to link one thing to another. In machine learning, it maps keys to values and updates itself to improve this mapping. A key could be a token, a gradient, or a hidden state, and a value could be an embedding, a momentum vector, or an attention value. In this case, training becomes a compression problem: just take many key→value pairs and store them efficiently in parameters.

For example, when we train a 1-layer Multilayer Perceptron (MLP) using gradient descent, the model is basically learning a mapping: input → signal (the gradient). The gradient update is the solution to a small optimization problem over the weight matrix, so the weight matrix is itself an associative memory storing what the model has learned so far. This is a 1-level nested system.

How do we get two levels?
→ Nested Learning uses a structure where different parts of the optimization process update at different speeds.

Fast memory: the inner optimizer state that updates every step and captures short-term learning signals.
Slow memory: the model weights that update more slowly by integrating those fast updates.

In this two-level system, the inner optimizer learns rapid patterns, while the outer optimizer consolidates them into long-term parameters.

Transformers add their own fast memory through attention, which recomputes token-to-token relationships on every forward pass.
Attention updates instantly, optimizer states update every step, and weights update slowly – giving the model multiple natural time scales even before NL.

Nested Learning treats the optimizer as an explicit memory module:
momentum stores recent gradients, Adam stores two moments, and more advanced optimizers hold richer internal states. NL generalizes this into a structured hierarchy where each level learns at its own speed.

If we sort components by how often they update, we get a hierarchy of levels, which forms the structure of Nested Learning:

A neural network becomes a Neural Learning Module – a hierarchy of memories, each optimizing its own internal objective at its own frequency.

But if we want continual learning, we need multiple levels of memory, each with its own update frequency, not just “short-term vs long-term.” This idea led to the Continuum Memory System (CMS) and then to the new HOPE architecture.

Instead of one MLP (long-term memory), Continuum Memory System (CMS) uses multiple MLPs, each updating at a different frequency, for example:

Each MLP in the CMS’ chain is indexed by its update frequency and has its own parameters, so they update on different schedules. This creates a continuum from fast to slow memory:

In this paradigm, Transformer is a special case with only one MLP that is updated only during pretraining. That is why Transformers are mostly static, which doesn’t fit with the idea of continual learning at all. This encouraged Google Research to propose another variant of architecture for this purpose.

All of Google’s new continual learning ideas meet inside one system called HOPE: a Hierarchical, Optimizing, Persistent, Evolving module. It’s a self-referential learning system built around three core elements:

While a Transformer has only 2 memory systems (attention as fast memory, and MLP as slow memory that never changes after pretraining), HOPE is the system of another level with notable benefits:

In addition to this, HOPE architecture achieves lower perplexity and higher accuracy than Transformers and other recurrent models, such as Samba and Google’s own Titans. It also handles extended sequences more effectively.

Though the Nested Learning paradigm and HOPE feel new and highly optimized, they come with their own challenges:

Overall, Nested Learning is a fresh variant of continual learning, and HOPE is a great shift to implementing it in a real architecture. Google’s perspective shows how optimizers, attention, and weights all fit under the same learning architecture, proposing something that we haven’t seen earlier.

Another outstanding approach is a more universal one and is focused on solving the main problem of continual learning: How can a model keep updating its parameters without breaking everything it learned before? Let’s see →

Researchers from FAIR at Meta propose their solution: memory layers. These are high-capacity layers where only a small number of parameters are active on any given forward pass. The key feature is that updates stay sparse and localized, so they don’t overwrite the model’s core knowledge.

What is a memory layer basically? This layer replaces one of the Transformer’s feedforward networks (FFNs) with a sparse attention lookup into a giant pool of learned keys and values. A memory layer adds a huge table of learned “memory slots” – there can be millions of them. Instead of running a normal MLP, the model queries this memory to pull out the most relevant stored information.

Here is how it works step-by-step:

Here keys and values are parameters, not activations as in standard attention: keys learn when a memory slot should fire, values learn what useful information to output. As a result, the memory acts like a trainable cache inside the model.

FAIR at Meta researchers followed the key idea: since memory layers are sparse, we can update only a tiny number of parameters when the model learns something new. It is the fundamental idea behind their method called Sparse Memory Finetuning which gives both:

How to find memory slots that are specific to the new fact the model needs to learn?

Researchers borrowed an idea from classic information retrieval: TF-IDF (Term Frequency-Inverse Document Frequency). It measures how important a word is in a document, compared to how common it is across all documents. TF-IDF is high when a term appears often in one document but rarely elsewhere.

In the case of sparse memory finetuning, TF-IDF:

A slot that is frequently accessed for the current example but rarely accessed elsewhere is treated as highly specific. This means it is likely storing information unique or highly relevant to the new fact.

Then only the small set of high-TF-IDF slots is fine-tuned, while everything else (most of the model) stays frozen.

In experiments, this approach dramatically reduced forgetting. When teaching models new TriviaQA facts:

To sum up, here is why memory layers are such a strong fit for lifelong learning:

But here are several…

Despite this Meta’s approach is much more universal and easy to implement in current models.

Today’s models still live inside the moment. They operate on the immediate present, while older memories stay frozen in time. That stops them from evolving, because they can’t reliably build new knowledge or new skills during real use. Humans learn through their whole lives, so models should keep learning after deployment too. We’re entering a phase where inference matters more than training. This capability is the missing ingredient, even in the most capable reasoning models.

It provides a balance between:

Stronger lifelong learning would help us build AI that behaves more like humans, correct biases or errors in deployed models without full retraining, and enable on-device learning where constant retraining is impossible. Fortunately, more researchers are now focusing on continual learning and exploring ways to integrate this missing puzzle piece into AI systems.

Nested Learning and sparse memory finetuning push this shift forward. Sparse memory finetuning gives existing systems like Transformers a universal path to adaptation, helping them absorb new knowledge while keeping their older memories stable. Nested Learning adds a new lens on neural networks by treating them as multi-level stacks of memories that learn at different speeds. It ties together many parts of deep learning and opens a path toward more expressive, stable, and biologically inspired learning systems. And this is only the first chapter of a much bigger story.

Sources and further reading