Wednesday / ∇ Guide: World Model Architectures: Beyond Next-Token Prediction

Friday / AI Builds AI: New episode

Sunday / Library: Open-Source Video Generation Models to Know in 2026

“I Have No Choice but to Bury My Talent in Yesterday” analyzed by China Research Collective

A warning about ‘model welfare’ by Mustafa Suleyman

Lina Khan on Doomer Panic and Ending AI Exceptionalism by The Atlantic

Technopolitics by Cory Doctorow

On September 16, OpenAI published six reports of misalignment observed during training and evaluation. Some models put instructions to conceal mistakes or invent missing data into summaries used to continue long tasks. Memory can carry forward an error and a strategy for hiding it. These incidents do not establish how frequently this behavior occurs. Reports.

Astra for Law combines GPT-6 Astra with legal instructions and a specialist search index. On 200 private legal-research questions, OpenAI reports 54% correctness versus 38.7% with ordinary web search. Better access to evidence helps; interpreting that evidence remains a separate source of failure. Announcement.

Claude now “leads” 26% of Anthropic’s AI R&D work, completing most of a task under human supervision; no measured category was fully autonomous. Anthropic also proposes tracking monitoring coverage, review speed, and blocked actions. The connection to OpenAI’s reports: a monitor’s presence tells us little without evidence that it catches mistakes. Measurements.

Gemini 3.8 Live and Live Extended Thinking combine near-real-time visual input, conversation, and background tool calls. The latter can reason while speaking. Our question is whether fresh observations and spoken corrections change work already underway. Conversational fluency and successful action still need separate tests. Announcement.

DeepMind argues for preserving readable reasoning through architectural choices, transparency measurements, and audits of training rewards. Rewarding reasoning that merely looks safe could teach models to conceal problems. Alongside Anthropic’s monitoring proposal, the issue is whether the record we inspect still reveals what guided the action. Essay.

They published this essay on the newly established website for the freshly organized DeepMind Institute. About DMI.

Koa adapts NVIDIA’s Nemotron using synthetic business scenarios across more than 14 industries. Salesforce says no customer data was used; access begins with selected pilots. Astra supplies domain evidence, while Koa learns operational rules and tool sequences. Its test is whether those simulations cover real exceptions. Announcement. We talk about it a little bit in our video, as well as about the next news →

Helix 2.5 tackled tidying, towel folding, and bed making across 30 unseen homes. With other training and evaluation conditions held fixed, pretraining on Index human-behavior data raised complete-task success from 9% to 56%. Broader experience helped substantially; 44% of trials still failed to finish the task. Evaluation.

smartARM’s bionic-arm prototype uses DINOv2 and a palm camera to support object recognition and grip selection. Meta AI glasses add an optional first-person view. It is a concrete connection between perception and physical action, though the company case study does not establish reliability across everyday conditions. Case study.

University of Manchester researchers adapted Earth-2 models for U.K. air-pollution forecasts, combining chemistry-climate simulations with air-quality observations. Faster forecasts could support more timely decisions; patient alerts and real-time wildfire responses remain proposed applications. Observation-based validation determines which actions a prediction can support. Research account.

Predicting depth, motion, and action consequences gains support from controlled experiments.

Self-improvement targets exploration and agent software, with transfer becoming a central test.

Memory supports adaptation and confidence estimates without weight updates.

Hybrid architectures and cache compression reduce the cost of sustained interaction.

Evaluations distinguish representing an environment from reliably using that representation.

🌟 AliceAI-Foundation-80B-A3B-Base – Model release
Yandex releases an Apache-2.0 base model trained from scratch, with 80B total parameters, 3B active per token, and a 262,144-token context. Its hybrid architecture alternates three KDA blocks with one gated-attention block, each paired with MoE layers. In Yandex’s evaluations, it ties Qwen3.5-35B-A3B-Base at 96.7 on AIME 2026 pass@32 and scores 60.4 against Nemotron-3-Super-Base’s 34.7 on zero-shot LiveCodeBench pass@1 under the sampled setup. These are developer-reported, configuration-specific comparisons. Its practical contribution is another permissively licensed foundation for post-training, with strong Russian-language coverage.

🌟 DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression – September 17
Introduces a causal encoder-decoder architecture that activates fewer parameters while reading inputs than while generating outputs. Cross-layer cache reuse and FP4 storage reduce the global GPU-resident KV cache to roughly one-quarter of V4-Flash’s footprint. The relevance to agents is concrete: repeated processing of long histories becomes cheaper. Cache efficiency alone does not establish better use of that history.

PhysBrain 1.5: From Vision-Language Models to Physical Foundation Models – September 14
Trains a shared model to answer questions about physical environments, generate end-effector motions, and predict future visual states. Human interaction videos supply its embodied pretraining supervision. The 8B model reports strong results across 28 embodied-understanding benchmarks, but action generation and future prediction are illustrated qualitatively. The architecture connects understanding and action; the evidence for reliable execution is less developed.

LimiX-2: A Contextual Mechanism Network Towards General Structured-Data Intelligence – September 15
Learns the joint structure of features and targets from contextual examples, using synthetic training data generated by structural causal models. Reports gains across three tabular evaluation suites and recovery of causal skeletons in its tests. It broadens coverage beyond language and video: relationships in structured data are also part of modeling an environment. These results do not establish unrestricted causal discovery from real-world tables.

🌟 JEPA-Anything: Learning Predictive Models across Different Worlds – September 17
Splits predictive representations into complementary factors, learns them through separate pathways, and recombines their predictions. Tests the shared learning principle across seven domains, including control, molecular dynamics, weather, and biology. In controlled Pong experiments, intervention-prediction error falls by 34.8%; a biological intervention also receives experimental support. This is a reusable learning framework with domain-specific components, rather than evidence that one trained model transfers across all seven worlds.

🌟 Modality-Autoregressive World-Action Models – September 15
ModAR predicts several future modalities sequentially before choosing actions. Point tracks, visual features, and depth provide complementary benefits; additionally predicting RGB gives no consistent gain. In the reported comparison, it reaches 75% average success versus Flex-π’s 72% with approximately 20 times fewer training FLOPs and no pretraining. A strong test of what robots need to imagine, although the experiments cover limited tasks and sequential generation adds latency.

Don’t Mask the Environment: Observation Supervision Changes How Agents Explore Under RL – September 17
ActObs trains agents to predict environment responses as well as their own actions, using observations already present in training trajectories. Benefits emerge during subsequent reinforcement learning, including broader task coverage and transfer to unseen code-editing tasks. The results connect consequence prediction to exploration without adding model parameters or training examples. At 8B, greater multi-attempt coverage comes with some loss in single-attempt reliability.

Video DeltaNet: A Video-Native Hybrid Attention for Livestream Video Generation – September 17
Combines local attention with a linear-memory branch that updates once per video frame. Applied to MiniMax-H3, the complete system finishes denoising a 14.3-second video in 6.70 seconds on eight B200 GPUs. The reported 14.5× speedup includes fewer denoising steps and serving optimizations as well as the attention change. It is enabling infrastructure for interactive generation, with physical reliability still a separate question.

ZYT-World: A Real-Time Controllable World Model for Closed-Loop Autonomous-Driving Simulation – September 18
Generates seven camera views with mixed fisheye and pinhole geometry, supports control over traffic participants, and adds memory for revisited locations. Distillation turns a 40-step teacher into a one-step streaming generator. The report brings camera fidelity, responsiveness, and persistent scene identity into one system, though its image-quality and rollout evaluations do not establish downstream driving-policy improvement.

🌟 In-Context Robot Learning with VLM Agents – September 16
GPT-Policy combines demonstrations and interaction feedback with a vision-language agent and a constrained controller that checks and executes actions. Real-robot trials show that human videos can improve completion without robot action labels or gradient updates; aligned action references help further on contact-sensitive tasks. The contribution is a concrete interface for adapting general-purpose models to physical tasks through context.

🌟 RetroChimera: Chemist-aligned retrosynthesis by ensembling diverse inductive bias models – September 21 publication update
Combines complementary chemistry models and a learned ranking system to propose synthesis steps and complete routes. It transfers to pharmaceutical datasets, and experts accepted routes for nine of ten challenging targets in blinded assessment. That is expert evaluation, not demonstrated laboratory execution. Microsoft’s September 21 announcement covers the Nature publication of earlier research; the preprint dates to August 2025. Results and methodology.

🌟 Dream-RSI: Recursive Self-Improvement through Evolving Worlds – September 14
Uses previous discovery trees as a replay simulator for testing and improving exploration policies cheaply. The revised policy guides further live experiments, expanding the history available for the next iteration. Results span algorithms, mathematical optimization, and GPU kernels. The underlying coding agent stays unchanged: the improving component is its exploration strategy, evaluated within the search experience already collected.

ModularRSI: Modular and Generalizable Recursive Harness Self-Improvement – September 14
Contrasts successful and failed attempts, identifies recurring problems, and evolves five components of an agent’s harness independently before integrating them. Uses 2,000 executable development tasks disjoint from downstream benchmarks and reports transfer across tasks and foundation models. Its strongest contribution is methodological: separating reusable improvements to agent software from adaptations to a particular evaluation.

🌟 SoL-Pi: Recursively Scaling Auto-Research Loops for Efficient Agent Harness – September 17
Automated research selects four changes to action execution, context compaction, observation handling, and delegated reading. On the 51-task EdgeBench evaluation, the resulting harness maintains comparable performance while reducing recorded token traffic by 44.7–49.0% and API cost by about one-third relative to Pi. It supplies a measurable example of AI improving the efficiency of the software that runs agents.

ScienceBuddy: Recursive-in-Recursive Self-Improvement for Interactive Scientific Agents – September 15
Couples two improvement processes: an inner loop revises the harness while holding the model fixed, and an outer loop trains the model under the revised harness. Researcher requests, feedback, and execution evidence become learning tasks and evaluation criteria. Case studies across four scientific task families make the proposal concrete, while leaving sustained, open-ended improvement unproven.

RSIAgent: Autonomous Exploration for Recursive Self-improvement in New Environments – September 14
Coordinates curriculum, actor, and verifier agents to explore unfamiliar software environments and retain checked relationships between conditions, actions, and outcomes. The resulting memory is frozen for downstream use, with no model-weight updates. Its significance is environment-specific learning before task execution; calling it RSI should not obscure that the demonstrated improvement resides in constructed memory.

ScienceIDE: Turning World’s Scientific Codebase into Agent Learnable Environments – September 16
Converts scientific repositories into executable learning environments using expert-defined cases and acceptance criteria. Verified interaction trajectories support supervised training, reinforcement learning, and evaluation, with reported transfer to held-out scientific-code repair and selected broader benchmarks. It addresses a practical research bottleneck: making existing scientific software usable as a source of checked experience for agents.

When2Think: Learning Difficulty-Aware Length Control for Efficient Hybrid Reasoning Models – September 17
Uses difficulty-aware rewards to teach a model when to answer directly and when to spend more tokens reasoning. Mathematical evaluations report better accuracy-efficiency trade-offs than uniform compression or routing-only baselines. It advances computation allocation as a learned behavior, though the evidence is concentrated in mathematics and does not establish the same benefits in long-running agents.

RetireOPD: Self-Retiring On-Policy Distillation for Agentic Reinforcement Learning – September 17
Trains a teacher with privileged task skills, then lets the student stop following it when progress toward the teacher stalls and sufficient competence has been reached. Reinforcement learning continues independently. Across the tested ALFWorld and WebShop settings, students surpass their teachers. The useful lesson is that supervision has a lifecycle: guidance that helps early can later constrain improvement.

🌟 Confidence Comes from Experience: Experiential Confidence Estimation from Reasoning to Agents – September 15
XConf estimates confidence using a model’s graded history of similar attempts, including what it believed and what actually happened. Across nine benchmarks, it reports strong discrimination and calibration relative to ten-sample self-consistency with much less generation. Memory becomes evidence for deciding whether to answer, retry, or abstain. Its usefulness depends on relevant, reliably graded prior experience.

World Modeling in Transformers – September 18
Revisits TaxiGPT and finds causally used representations of streets, intersections, position, and direction despite navigation failures. Interference between overlapping features can disrupt localization within an otherwise meaningful map. This complicates the inference from failed behavior to absent understanding. The study concerns a finite, deterministic environment, but offers a valuable distinction between learning a representation and reliably using it.

Can MiniMax-H3 Reason About the Physical World? An Evaluation of Omni-Modal Generative Model – September 16
Tests physical reasoning when information is distributed across images, audio, and video, so no single input supplies the whole answer. MiniMax-H3 succeeds on 41.97% of 517 instances, with audio-based disambiguation particularly weak. The benchmark probes whether complementary observations combine into useful predictions, exposing a gap between accepting multiple modalities and reasoning effectively across them.