World models are generative AI systems designed to capture how our 3D reality works. From diverse data, they learn the underlying physics, spatial relationships, and cause-and-effect of the world – then use that understanding to predict what happens next, run internal simulations, and make decisions without constant real-world testing.
World models remain a small but highly promising field. Each new development offers a glimpse into how AI is learning to model the physical world and the logic of action itself. We’re tracking these breakthroughs to keep you ahead of the curve.
In our previous articles about world models, we explained the basics – what they are and how their main examples work and an alternative vision on building world models with Physical, Agentic, and Nested (PAN) system. Today we’ll take a look at:
Meta’s groundbreaking Code World Model (CWM), which explores how world models can connect with the world of code and introduces a new reinforcement learning strategy by modifying GRPO;
Probabilistic Structure Integration (PSI) from Stanford NeuroAI Lab – a promptable, probabilistic world model where structure becomes new vocabulary.
And we’ll also briefly cover updates to Dreamer 4, Genie 3, and Cosmos WFM 2.5. Time to explore some exciting new tech!
In today’s episode, we will cover:
Code World Model (CWM)
CWM’s architecture and training
Special Reinforcement Learning Strategy (and more about four RL environments)
Practice and limitations
Probabilistic Structure Integration (PSI)
PSI self-improving workflow
Possibilities that PSI opens for us
Limitations
Other notable world models
Conclusion
Sources and further reading
Code World Model (CWM)
Let’s start with the model that played a part in the global debates about whether GRPO works properly. We’ll turn to GRPO and RL a little bit later, but firstly – what’s the idea behind Meta’s new world model and how does it refer to code?
Meta’s FAIR CodeGen team has extended the idea of world models into a domain that hasn’t traditionally been part of that conversation – code. LLMs and code have long been a natural pair, but in most cases models treat code as plain text: they generate it, fix it, or explain it, without understanding what happens when the code runs or how it changes a system’s state. This gap limits their ability to produce reliable, high-quality code that truly works.
Meta’s latest development, Code World Model (CWM), addresses that gap by bringing the practical, executable side of code into the model’s reasoning process.
CWM is a 32-billion-parameter model trained not just on static code, but also on data that captures how code behaves when executed. This allows CWM to keep on track how each line changes variables and how edits affect the whole program, so debugging, testing, and reasoning about programs go to the next level.
How is it organized from the technical side?
CWM’s architecture and training
As we’ve mentioned, CWM is a 32-billion-parameter decoder-only Transformer with 64 layers, a hidden size of 6144, and 48 attention heads. It uses an alternating pattern of local and global Sliding Window Attention (SWA):
Local SWA (8k tokens) handles short-range dependencies.
Global SWA (131k tokens) captures long-range context across large codebases or reasoning chains.
This pattern repeats 15 times throughout the model, providing CWM with 131k-token context window.
Image Credit: CWM original paper
For higher efficiency, CWM implements Grouped Query Attention (GQA) (speeds up attention), SwiGLU activations (for learning complex patterns), RMSNorm normalization (improves stability and training speed), and Scaled RoPE positional encoding (lets the model understand token order over very long sequences).
Llama 3 tokenizer, with extra reserved tokens added, allows for special reasoning and trace formats. It can run inference on a single 80GB NVIDIA H100 GPU thanks to quantization.
Now, let’s see what are these special reasoning and trace formats.
CWM’s pre-training goes through two stages:
First comes general pre-training on 8 trillion tokens of general text, code and STEM data.
In code world model mid-training stage, CWM gains its “code world modeling” abilities by training on 5 trillion tokens of specialized data. It learns from two types of observation-action data:
Python execution traces: They show how running a line of code changes program state, including variables and memory. They include over 120M traced Python functions, 70K traces of correct and incorrect code solutions, 21K repositories, and 75M natural-language traces. This gives the model a kind of “movie” showing what happens inside the computer during code execution and teaches CWM to simulate how Python code executes and to reason about program behavior.
Agentic interactions: CWM watches an AI ForagerAgent that acts as a software engineer fixing and modifying real software in Docker environment. This agent writes and edits code, runs commands, and reacts to feedback.
Overall data mix in this stage includes: 30% CWM-specific, 40% general code, and 30% rehearsal of pre-training data to prevent forgetting.
Next comes post-training stage, leveraging Supervised Fine-Tuning (SFT) and joint reinforcement learning (RL). RL strategy used in this model took part in triggering the buzz around RL and Group Relative Policy Optimization (GRPO) last week, so let’s take a deeper look at what makes it really interesting.
Image Credit: CWM original paper
Importantly, Meta doesn’t use RLHF (Reinforcement Learning from Human Feedback) for CWM training, since it is not meant to be a general chatbot. And that’s a good hint – broader reasoning needs something more that familiar RL schemes.
SFT is the preparation stage for RL, where CWM trains on 100B mixed tokens – public and internal instructions plus 30% rehearsal and reasoning data (OpenMathReasoning, OpenCodeReasoning). It strengthens reasoning and instruction-following, and a constant learning rate (1×10⁻⁵) and a 32k-token context balances stability and readiness for later RL. Special tags <|reasoning_thinking_start|> and <|reasoning_thinking_end|> mark reasoning steps, so the model can switch between normal (concise) and reasoning (step-by-step) modes.
Then comes RL stage with the main issue: What doesn’t work well in RL, and particularly in GRPO, that Meta decided to fix in their new world model?
Special Reinforcement Learning Strategy
For RL, CWM uses an improved version of GRPO, where it estimates returns directly using Monte Carlo value estimation. It adds several modifications to GRPO to make RL fast, stable, and better for multi-turn coding and reasoning tasks:
Supporting full multi-turn interactions like back-and-forth with an environment. (GRPO originally handled single-turn “prompts → response” environments.)
Asynchronous training: It runs data generation and training in parallel for higher throughput.
Fairer reward scaling: Removes standard deviation and length normalization to avoid biases toward easier or longer problems. Rewards are averaged and normalized by the maximum sequence length – 131k tokens.
Smarter batching: Groups data by a maximum token count rather than a fixed number of trajectories, keeping batch sizes more stable.
Higher clipping range: Uses slightly looser PPO clipping to preserve exploration, and so →
No KL penalty needed as the clipping setup already keeps entropy balanced.
Clean data filtering:
Skips zero-advantage to reduce noise and stale trajectories that are too old to reflect the current policy.
Applies weighted returns to handle long, hard examples fairly.
Uses a gibberish detector that rejects nonsensical outputs based on token rarity and probability.
Generally, CWM is trained across four RL environments:
Agentic SWE (software engineering): CWM acts as a coding agent that autonomously fixes real software issues through long, multi-step interactions (up to 128 turns and 131k tokens) using a self-bootstrapping process to train on its own high-quality reasoning traces.
Image Credit: CWM original paper
The coding RL environment focuses on competitive programming. CWM receives a problem, reasons with <think>...</think> tags, writes and debugs code (Python or C++), and earns rewards.
The agentic coding setup merges competitive programming with SWE-style tool use: running, creating, and editing code via tools like “bash,” “create,” and “edit.”
The math environment teaches CWM symbolic and numerical reasoning.
In the final phase, CWM is trained jointly across all these RL environments, using asynchronous RL setup:
Image Credit: CWM original paper
Worker nodes generate multiple trajectories (problem-solving runs) from these environments.
Trainer nodes receive them via a queue, form batches, and update the model. Some training batches also come from SFT data for rehearsal (~1/3 of the total), helping the model retain previous knowledge.
The joint RL task mix is 40% software engineering, 40% competitive programming, and 20% mathematics. This blend, with gradually rising complexity and controlled response length, teaches the model to generate efficient, high-quality reasoning and code across tasks.
If we zoom out from this asynchronous RL training setup, we see the same concept working at scale in CWM through the asynchronous distributed RL framework which makes that logic run fast and continuously on real GPUs. It separates data generation and training for keeping tens of thousands of GPUs and environments busy 100% of the time:
Image Credit: CWM original paper
Workers continuously generate trajectories – sequences of interactions between the model and an environment.
Trainers receive these rollouts, compute gradients, and periodically broadcast new model weights back to workers. In other words, they handle optimization.
Communication happens via transfer queues, so no one waits on anyone. Workers can also continue generating text using slightly older weights (the KV-cache remains valid), ensuring GPUs are never idle.
There is such a vast bunch of innovation in CWM, that the results don’t take long to show up →
Practice and limitations
In practice, CWM turns into the machine that works like any typical world model but in coding field. It can “mentally” run code without executing it on a computer, like “inner Python interpreter.” Its functionality range includes:
Writing and testing its own programs.
Spotting mistakes or logic errors.
Reasoning through problems by connecting code to its effects.
This new capabilities for code models could lead to tools like a “neural debugger” that predicts future program states, explains bugs, or generates code, reasoning about its expected behavior. This is reasoning grounded on practice.
Even aside from its world-modeling features, CWM performs strongly on standard coding and reasoning benchmarks:
Software Engineering (SWE-bench Verified): 65.8% (with test-time scaling)
Image Credit: SWE-bench Verified
Coding reasoning (LiveCodeBench-v5): 68.6%, competitive with Qwen3-32B’s 65.7%.
Mathematics: Math-500: 96.6%, AIME 2024: 76.0%, and AIME 2025: 68.2%.
Execution trace prediction: CruxEval output: 94.3%.
These CWM’s abilities are comparable to existing open models like Qwen3-Coder-480B, Llama 4 Maverick, and GPT-oss-120B.
Taken together, this shows that world modeling can make code generation more intelligent and reliable by simulating Python execution step by step.
However, CWM is too early for a broad use, because of these current limitations:
World modeling data focuses on explicit Python execution only.
It’s optimized mainly for code reasoning and generation, and is weaker on factual knowledge, natural language tasks, or multilingual content.
CWM is definitely not a general-purpose conversational assistant.
Now it’s a research-only release.
But the main issue is the methods for effective using of world models for planning, reasoning, or fine-tuning – they are too early and still remain open research questions.
That’s why the next major development we’ll explore pushes general-purpose world models further, giving them a clearer roadmap for how the world works. Let’s unpack the concepts behind it.
Probabilistic Structure Integration (PSI)
It’s no surprise that some of the most exciting breakthroughs in world modeling are coming from Stanford – the same place that gave us ImageNet, helped kick off the modern era of computer vision, and is now pushing the boundaries of video understanding and causal intelligence through its NeuroAI Lab.
Stanford NeuroAI Lab recently stepped into the field of self-Improving world models with better control and automatic unsupervised learning of structures. All of these aspects are combined in their self-improving framework for world models called PSI – Probabilistic Structure Integration.
PSI turns a video world model into a random-access probabilistic graphical model you can prompt like an LLM, except the “language” is local patches, pointers, flows, depths, and segments. That gives you precise handles for factual rollouts, hypotheticals, and counterfactuals, like “move this object right,” “pan camera up,” “reveal patches here”, and explicit uncertainty maps for perception and planning.
At its core, PSI learns, builds new structured knowledge like motion, depth, and object relations, and feeds it back to improve itself. These extracted structures become things you can prompt or condition the model with. In opens new control handles, when everything becomes meaningful. For example: before you could only say “show a person walking,” and now you can guide how fast they walk or what direction they move by adjusting the motion or depth tokens.
Here is what provides these capabilities from the technical side.
PSI self-improving workflow
PSI rebuilds the base model itself as a distributional, promptable world model, using a three-step loop:
Image Credit: “World Modeling with Probabilistic Structure Integration” paper
Probabilistic prediction (Ψ):
PSI learns a probabilistic predictor Ψ on raw video data. It break it into spatiotemporal patches (tokens) and trains Ψ as a random-access autoregressive model that can predict any patch when given any other subset of patches.
To make this possible PSI uses a Local Random-Access Sequence (LRAS): pointer token address every image/video patch; content tokens encode what is there. Interleaving pointers with content turns 2D/3D video into a 1D causal sequence yet allows arbitrary conditioning orders – sequential, parallel, or hybrid. That’s what the “random access” means.
Image Credit: “World Modeling with Probabilistic Structure Integration” paper
A Hierarchical Local Quantizer (HLQ) keeps tokens strictly local, so small edits have predictable effects and controls stay precise.
This way, Ψ model learns full conditional distributions, so it can model uncertainty, multiple futures, and flexibly answer “what if” questions (counterfactuals).
Extracting mid-level structures
This is where causal inference takes place. By prompting Ψ with counterfactuals (like moving a patch, shifting a viewpoint, other “what if” interventions ), PSI can reveal hidden structures such as:
optical flow (how pixels move together over time)
depth (how viewpoint changes cause parallax)
segments (groups of pixels that share motion)
This happens zero-shot and with no extra labels.
Image Credit: “World Modeling with Probabilistic Structure Integration” paper
Integration:
It’s the stage when control finally expands.
PSI turns the extracted signals into new token types (flow/depth/segments), and integrates those structures back into the same model, mixing them into sequences with the RGB tokens. The training continues from the stable checkpoint.
Image Credit: “World Modeling with Probabilistic Structure Integration” paper
Now Ψ can read and write these new signals, for example, predict RGB from flow, depth from camera motion, or compose them. Each cycle – pixels → physics → better scene understanding – expands the “language” you can use to control and question the model, and in turn, makes its predictions sharper and more useful.
This really feels like a breakthrough that could change how all world models work. But what can we actually do with it in practice?
Possibilities that PSI opens for us
By integrating new extracted structures back into the model, PSI gains dual ability:
Perception: Understanding what’s happening in data (e.g., recognizing moving objects), and
Prediction of what will happen next (e.g., forecasting future frames in a video).
It can connect different types of information – visuals, motion, sound, or text – into one coherent “world model.” And that’s truly important for general-purpose AI.
PSI allows for richer, more precise control. You can drive motion directly by supplying sparse flow on objects or by turning a desired camera move into a flow field. This beats specialized systems on novel-view synthesis and on 3D object manipulation.
You can now predict motion in its native space, segmentation stops confusing shadows or lighting with actual movement, and depth boundaries get sharper.
Adding a motion step (optical flow) before rendering the pixels makes the 500 ms–ahead prediction look closer to the ground truth than RGB-only baselines.
Here are some interesting examples that better illustrate what PSI allows one to do in practice:
Physical video editing: Small edits like moving the bowling ball, and recomputing consistent futures (e.g. pins fall or don’t).
“Visual Jenga”: Prompting tiny motions to measure asymmetric object dependencies and removing the safe block.
Robotics planning: From a single frame, you can get “what-will-move” maps and expected displacement fields for grasp/planning.
This is already quite a lot for the next step of world models’ evolution, but PSI also points to some open questions.
Limitations
PSI shows integration gains mainly for a local quantity (flow), but leveraging global structures, like segments to build object-centric predictors and jointly integrating multiple intermediates remains open.
Causal inferences at the extraction step is domain-dependent and not yet specified for non-vision data.
Automating the choice of intermediate structures is also unsolved.
Causal extraction is computationally heavy, and it could be lighter and faster with distillation of these signals.
Scaling PSI to long-range prediction likely requires online/adaptive memory mechanisms.
Achieving PSI’s distributional behavior generally needs substantially more parameters than, for example, CWM.
PSI is great at physical and structural stuff (flow, depth, segments), but it doesn’t yet build explicit category tokens or a built-in label space. So, PSI can precisely move a segmented object, but it doesn’t inherently know what the object is.
When overcome, these limitations could become a major growth point for PSI and for the entire range of world models. And what about them?
Other notable world models
Today we’re talking about what’s new in world models, which also includes updates to the main ones we covered in our previous article. They now feature a new tech stack aimed at improved performance and new brighter achievements.
Dreamer 4 from Google DeepMind
It is a the fourth-generation “Dreamer,” presented in September 2025 by Google DeepMind. Dreamer 4 is a world-model-centric RL system that learns and plans entirely within its learned model of the environment.
Its standout result is long-horizon control: Dreamer 4 is the first agent to achieve “obtain a diamond in Minecraft” task using zero online environment interactions, relying purely on offline data and imagined trajectories. This means the agent learned a 20,000-step-long task – mining a diamond in an open-world game from raw pixels, sparse rewards and mouse-keyboard control – by training inside its own world model. Its success rate is 0.7% over 60-minute episodes.
Image Credit: Dreamer 4 original paper
Architecturally, Dreamer 4 uses a highly efficient transformer-based architecture with a “shortcut-forcing” objective, enabling real-time simulation of complex 3D environments on a single H100 GPU. It also introduced a way to leverage vast amounts of unlabeled video:
It learns general environment dynamics, object interactions and action effects from gameplay video data.
Then it fine-tunes with a small amount (like 100 h out of 2,541 h) of action-labeled data.
This is also how, with Dreamer 4, Google DeepMind set a milestone for model-based RL.
Google DeepMind’s Genie 3
Genie 3 is a foundation world model announced in August 2025, build on the prior Google DeepMind’s Genie 1 and 2 systems. It is a generative environment simulator: given a text prompt, Genie 3 can create an interactive 3D world in real time at 24 FPS, maintaining consistency for a few minutes at 720p resolution. It is akin to a generative AI game engine that can produce diverse, dynamic environments on the fly.
Image Credit: “Genie 3: A new frontier for world models” blog
It is DeepMind’s first world model to support real-time agent interaction in the generated world (previous models could generate videos, but not allow an agent to move within them in real time). DeepMind’s achievements in Veo 2 and 3 models contributed to better physics and coherence in video generation of Dreamer 4.
This world model enables training and evaluating agents in open-ended, text-specified environments. Based on descriptive prompts, Genie 3 can simulate a wide range of physical scenes – from navigating volcanic terrain with realistic vehicle physics to weather events like hurricanes.
Cosmos World Foundation Models 2.5 by NVIDIA
In September 2025, NVIDIA updated its Cosmos World Foundation Models to the 2.5 version. It is a suite of generative world models for simulating physical environments, primarily for robotics and autonomous vehicles. Cosmos WFM 2.5 is a public platform that lets researchers use state-of-the-art simulators without building them from scratch. So what new features came with the update?
NVIDIA mostly simplifies and strengthens the stack. The new Cosmos-Predict 2.5 is the “world maker”: it combines three WFMs into one unified model that turns text, images, or short videos into longer, higher-quality simulations, including multi-camera views.
Image Credit: Cosmos-Predict2.5
In this stack, Cosmos-Transfer 2.5 takes rough simulator signals (edges, segmentation, depth, blur or real video) and renders more photoreal, better-aligned video with less drift, while being ~3.5× smaller than the last release. This allows for easier setup plus better fidelity and control.
Conclusion
Each of these world models – from code execution simulators to visual predictive systems and interactive environments – has advanced its own domain. Together, they show steady progress toward AI systems that can imagine and reason about complex worlds, while exploring different ways to represent reality:
Using CWM as an example, we can see that code is a promising field for world models, enriching their understanding of the world of machines and algorithms.
PSI offers a glimpse of what “visual prompting” will look like when world models communicate in tokens for flow, depth, and segments – not just pixels.
Thinking about LLMs as the main path to AGI gives only part of the picture. LLMs reason through language and abstraction, but they don’t form an internal sense of how the world behaves. World models address that gap by learning physics, causality, and embodied interaction – the grounding that connects reasoning to reality.
They’re still moving forward step by step, expanding to capture a wider view of the world and to act more confidently within it. It remains one of the most promising areas in AI, with many questions still open and discoveries ahead.
Sources and further reading
Resources from Turing Post
