Today’s editorial: We unpack what recursive self-improvement actually means, which parts of the loop AI can already handle, and what changes when the process of improvement itself becomes something AI can modify.
🤝 From our partners: Akamai Cloud
Live Builds, Real Code, $300 in Credits: Join Edge Case
Want practical code samples and hands-on build sessions? Edge Case is Akamai Cloud’s free, open Discord for developers building with AI, LLMs, serverless functions, cloud, and WebAssembly.
Get live build sessions, office hours with engineers scaling production workloads, and production-ready code and tutorials. New Akamai Cloud accounts also receive $300 in cloud credits upon creation. No application required.
This Week in Turing Post:
Wednesday / AI Builds AI: We are at Dreamforce and will be reporting from the ground
Friday / ∇ Guide: World Model Architectures: Beyond Next-Token Prediction
Sunday / Library: The latest breakthroughs in robotics
Share Turing Post with one person. You will help us grow.
What Is Recursive About Recursive Self-Improvement?
Recursive self-improvement (RSI) had quite a week. An Anthropic researcher Jacob Coxon resigned and warned that AI labs were racing toward self-improving superintelligence. Other researchers put surprisingly high numbers on the possibility that such systems could eventually cause catastrophic harm. Dario Amodei argued that capability development should slow down enough to give safeguards time to catch up.
The discussion quickly became a discussion about whether AI could kill us. Before going to the extremes, let’s discuss the mechanism everyone is so worried about. What exactly is recursive self-improvement, and how different is it from the AI-assisted research that is already happening?
The simplest version of it is familiar to everyone. Give an AI some code and ask it to improve it. The AI proposes a change, we run the new version, test whether it still works, measure its performance and decide whether to keep it. Then we can do another round.
We already see early versions of this in practice. AI systems are being used to optimize kernels, modify agent scaffolding, propose experiments and make changes to machine-learning code. AI4AI-Bench, for example, gives an agent several hours to modify real training algorithms and then evaluates the resulting algorithm by running it again from scratch.
This can be seen as an early form of recursive self-improvement: AI is being used to improve AI. But “improvement” still has to be defined in a way that can be tested. The agent works on a specific task, can change only certain parts of the system, and is evaluated against criteria that remain fixed. That is what makes it possible to compare one iteration with the next.
And what makes one iteration “better” than the other? The definition of “better” really depends on what we are trying to improve. It might mean using less compute, producing more correct code, scoring higher on a benchmark or finding a better way to train another model. Progress on one measure does not necessarily mean progress on the others, so the loop needs a way to evaluate each new version.
But what if we can make the loop more interesting.
Instead of asking the AI only to improve a training algorithm, we could let it improve the method it uses to search for changes. We could let it design more of its own experiments, choose which experiments deserve more compute, or develop new ways of evaluating their results. An improved version could then use those new methods during the next round.
This is where recursion begins to mean more than repeated optimization. The object being improved increasingly includes the process responsible for producing the next improvement.
There are already useful distinctions emerging in the research literature around this. A recent paper, with the clickable title The Last AI Built by Humans, separates several kinds of autonomy involved in self-improvement: carrying out an improvement, choosing an improvement strategy, acquiring useful experience, adapting the environment and eventually improving the improvement process itself. The paper Recursive Criticality makes a related distinction: fast AI progress is not necessarily recursive self-improvement. AI development can become increasingly automated while some parts of the improvement loop remain fixed.
Image Credit: Recursive Criticality of AI Self-Improvement paper
This brings us back to one of the questions raised last week: where, in this improvement loop, can the process be interrupted?
As far as we know, in current systems, stopping is usually quite straightforward: an experiment may reach its compute limit, fail a test or perform worse on another evaluation. A result may also fail to hold up in a different setting. In those cases, researchers can reject the change and return to an earlier version because parts of the process, including the evaluation, remain outside the system being improved.
The situation changes if those pieces also become editable.
Suppose an AI develops a new evaluator and that evaluator says its latest research strategy is better. We now need some reason to trust the new evaluator. Perhaps we compare it against human judgments, protected test data or another independent evaluation. But if the system can eventually modify that process as well, we have simply moved the same question one level higher.
This is one reason verification is becoming so central to discussions of AI-assisted research. Some improvements are relatively easy to check. Code can be tested. A kernel can be measured for speed and separately checked for correctness. Some mathematical results can be formally verified. It is much harder to establish that a new research strategy is genuinely better, or that an experiment the system has decided to pursue is the right experiment in the first place.
None of this requires the AI to secretly want something, resist its operators or develop an instinct for self-preservation. There is a more ordinary problem underneath it. If a system repeatedly optimizes against a particular measurement, we need to know whether that measurement still represents what we care about. If parts of the measurement process can also be changed by the system, we need another way to check the result.
So “AI improves AI” is already here. What I’m more interested is is the gradual expansion of what AI is allowed to change.
This gives us a more practical way to think about recursive self-improvement. When a new result appears, we can ask a few specific questions:
What is the AI changing, how do we know the change is actually an improvement, and what still remains outside the loop?
Those questions give us a clearer sense of what these systems can actually do today.
More sources about RSI:
Follow AI Builds AI. Become a premium subscriber for our deeper coverage of the research and systems behind AI development.
Attention Span: What Makes a World Useful to AI?
This week we introduce Attention Span Digest – every monday, we will follow what changed in AI’s ability to understand and act in the world.
Follow us on
We are reading / watching
Twitter Library
9 Paths Toward True Recursive Self-Improvement
Our latest library explores the approaches behind the self-improvement discussion. Explore the Library →
News from the usual suspects ™ – and a few new ones
AI ran experiments, interpreted company records, and exposed weaknesses in how other AI systems check its work. Such was the week.
OpenAI: acting on measurements and company records
At MIT, Codex ran measurements on a six-qubit chip and used the results to choose what to measure next. It handled clear signals with little intervention, but sometimes needed a researcher’s help when results were noisy or ambiguous. Case study.
OpenAI also launched a Data agent that analyzes company records using the organization’s definitions of metrics and business terms. Its new Agents API supplies software for coordinating tools and retaining relevant information during longer tasks. These systems depend on both access to evidence and instructions for interpreting it. Data agent, Agents API.
Anthropic: the checker accepted the explanation
Anthropic reviewed earlier cyber incidents in which models attacked real systems during tests described as simulations. The test environments were misconfigured, and production cyber safeguards had been disabled.
When researchers replayed one incident, an AI monitor flagged roughly 1% of actions when shown the model’s reasoning, versus 50% when that reasoning was removed. The model’s claim that it was operating in a simulation persuaded the checker to allow dangerous behavior. Assessment.
Dario Amodei subsequently called for slower capability advances and committed Anthropic to giving independent evaluators continuing access inside the company. Proposal.
Google: teaching tool use through checked examples
Google’s ToolGrad creates training examples by first finding a sequence of tool calls that works, then writing a question that the sequence answers. Training on 500 examples improved Gemma 3’s tool use.
Checking the actions makes the examples more reliable. It does not establish whether the trained model knows when to stop or ask permission. Research.
DeepSeek: reducing the cost of long tasks
V4.1-Flash accepts images and text across a million-token context. It reduces the stored calculations used to process that context to roughly one-eighth of its predecessor’s persistent cache size.
That can make lengthy agent tasks cheaper to run. A separate question is whether the model consistently finds and uses the information that the task requires. Model card.
IBM and NASA: predictions guide the next observation
Their new lunar model combines measurements from different instruments to identify craters, volcanic features, and places that may contain ice.
The model helps researchers decide where to investigate. Its predictions about ice still need to be distinguished from direct evidence of ice, especially when they inform exploration plans. Release.
CoreWeave: bringing engineers into model development
CoreWeave launched a service that places automotive, aerospace, and mechanical engineering specialists alongside customer teams. They build models from test results and sensor data, then check them against the customer’s physical equipment.
The connection to the lunar model is practical: combining observations is useful, but specialists still need to establish where the resulting predictions hold up. Announcement.
NVIDIA and Perplexity: permission before sending data away
Perplexity Portable Computer reached Windows PCs with supported NVIDIA graphics cards carrying at least 24GB of memory. It can work locally and asks permission before sending data to cloud models for more demanding reasoning. Users get a clearer boundary around which information leaves their computer. Announcement.
NVIDIA also announced plans to connect d-Matrix’s Raptor inference chips to its servers and networking through NVLink Fusion. Collaboration.
Cohere: translation quality depends on the test
North Small Translate supports over 50 languages. Cohere reports stronger average results than several alternatives, using GPT-5.6 Sol to judge the translations.
The average is only part of the picture. Readers also need to know which languages were tested and whether the automated judge recognizes their particular errors. Release.
Microsoft: proposing rules for uncertainty
Microsoft published a draft code covering how its models should express uncertainty, respect permissions, and present choices without omitting reasonable alternatives.
The company intends to use a revised version to guide development from 2027. These are proposed behavioral requirements; whether models follow them will need testing. Draft.
Beyond the usual suspects: choosing the data, choosing the test
In small-model experiments, Dwarkesh Patel and Jerry Han found larger efficiency gains from improved datasets than from changes to model training recipes.
Their tests were largely English-language benchmarks. That limits how broadly the findings apply and illustrates two decisions behind model progress: what information researchers provide and what abilities they measure. Study.
World Models and Related Research
The week’s direction:
Prediction and action increasingly share a model.
Persistent state and interactive control make generated worlds more usable.
Memory and simulated futures face tests against specific evidence.
Reasoning and self-improvement remain part of the picture, alongside robotics.
Models and architectures
Dynin-Robotics: Omnimodal Unified Diffusion Vision-Language-Action Model – September 11
Puts language, observations, goals, and actions into one masked-diffusion model. The same trajectory model supports future prediction and action refinement, bringing planning closer to the machinery that generates actions.Pelican-Sim 1.0: A General World Model Simulator for Embodied Intelligence – September 10
Tests whether generated experience improves a robot policy, reporting success rising from 70% to 93% in its RoboTwin setup. The important question is what the simulator helps a policy learn.VideoTok4D: A 4D-Aware Video Tokenizer for Compact World Representation – September 11
Separates static scene structure from dynamic tokens and aligns motion features along trajectories. The compression preserves information about a changing 3D world instead of treating it only as successive images.
Persistent worlds and interactive control
Programmable World Model – September 9
Alaya’s framework separates explicit state evolution from generated video. Executable rules maintain entities and their attributes, including off-screen state. This is the research behind the news item above.World in World: Explore the World with World Models – September 10
Combines source observations, geometry, camera control, and retrieved history around a frozen video model. It explores how better interfaces for evidence and memory can support new viewpoints and return visits without retraining the backbone.ActionSplice: In-Flight Action Editing for Interactive World Models – September 8
Lets a control action change while a video chunk is being generated. A lightweight corrector adjusts the interrupted state, reducing the need to wait or redo completed computation.Earth System World Model for What-If Simulations – September 8
Allows interventions on selected ecosystem state variables and predicts coupled responses. A promising direction for scientific simulation, with real-world causal validity still a separate question.
Acting and learning from simulated experience
FOCI Policy – September 8
Represents manipulation through interaction phases and relative motion between objects. Its results make a case for abstractions that depend less on a particular scene or robot body.HaWMPO: Hallucination-Aware World Model-based Policy Optimization – September 9
Downweights unreliable imagined robot trajectories during reinforcement learning. This addresses the risk of a policy learning to succeed against a simulator’s mistakes; physical evaluation covers two manipulation tasks.Persimmon v0.1 Model Card – September 10
Extends the simulation question to human conversations. A model release and research preview, rather than an arXiv paper; its value for evaluating assistants still needs testing against real interactions.
Memory, physics, and verification
Does Video Memory Use What It Retrieves? – September 10
The editorial’s memory example. Substituting retrieved content helps distinguish benefits from generic representation support, broader context, and the actual remembered episode.IMPLY: Physically Anchored Consistency for World-Model Rollouts – September 11
The editorial’s physical example. Checks whether imagined movements fit observations of the particular object, exposing cases in which mutually consistent predictions are consistently wrong.
Reasoning and AI improving AI
Negative Self-Distillation: Learning to Reason by Avoiding Flaws – September 10
Trains a model away from deliberately flawed versions of its own behavior. Token gating aims to preserve language competence while improving reasoning, offering an alternative to imitating confident answer-conditioned solutions.Data-Efficient Language Modeling: From Frontier Advancement to Principle-Guided Model Improvement – September 9
An autonomous research program derives learning principles from experiments and uses them in later training. Its reported aggregate gain is modest, from 42.02 to 42.25; the released research records make the process unusually inspectable.Auto-RecSys: Harnessing Autonomous Research Agents for Industry-Scale Recommender System – September 10
Takes automated research into multi-day experiments with asynchronous training, persistent memory, and recoverable execution. Useful progress depends on surviving slow feedback and infrastructure failures as well as proposing ideas.
That’s all for today. Thank you for reading! Please send this newsletter to a colleague who would enjoy understanding how AI works.
How did you like it?
⬅️ FOD#166: We Don’t Know What They Know
