FOD#171: Why EmbeddingGemma 2 is so interesting

Wednesday / AI Builds AI: State of Agent Memory

Friday / ∇ Guide: 3D Gaussian Splatting: How do robots remember what's no longer in view?

Sunday / Library: Top research on Looped Transformers

Text and code use a 270-million-parameter component.

Vision adds 170 million parameters.

Audio adds another 300 million.

Does Claude Have Rights? by Mustafa Suleyman

Old one but good one: The neuroscience of memory - Ri Science Podcast with Charan Ranganath (video)

Investigating unintended model actions in our evaluations and internal use by Anthropic

Bending the Curve of Discovery: AI in Science Today and Tomorrow by Alex Imas and James Manyika

Models as Insider Risks in the Super Intelligence Era by Satya Nadella

Mathematics gets a very large reading assignment. OpenAI published mathematical results from an unreleased frontier model. The updated repository contains 719 manuscripts across 372 families, with roughly 42% of top-line results formalized in Lean. Verification is ongoing, so these should not be counted as 719 established breakthroughs. The scale nevertheless raises a serious question for mathematics: how quickly can researchers check, understand, and build on machine-produced results?

ChatGPT gains Intelligent UI. The October 7 GPT-6 announcement expands access and introduces responses assembled from interactive diagrams, charts, forms, and small tools. A native component library and streaming compiler let interfaces appear progressively. The model now chooses how to present an answer as well as what to say.

Watermarks and advertising expand. October 5 brought opt-in API text watermarking, with eligible EU ChatGPT and Codex outputs to follow, and a visual-ad format scheduled for US testing during image generation later this month. OpenAI says ads remain separate from generated images and do not influence answers.

Claude Haiku 5.5 arrives for high-volume work such as classification, summaries, database queries, and coding subagents. It is the first Haiku with adjustable effort. Anthropic reports average running costs around 75% below Haiku 4.5 and also halves Sonnet 5.5’s cache-read price.

Claude Dashboards and Motion enter beta. Dashboards connects to company data and exposes the queries behind its charts; Motion creates editable, code-based animations rather than generated footage. Docs, Slides, and Design leave beta and become available across plans, including Free.

The Anthropic Cyber Mission brings models, engineers, and support to critical-infrastructure defenders and open-source security. Its accompanying usage-policy update, effective November 12, clarifies restrictions around deceptive campaigns, weapons software, surveillance, and autonomous physical actions.

Google Cloud introduced the Gemini agent, a unified enterprise agent for research, coding, content creation, and multi-step work. It carries memory, skills, and controls across Workspace applications, supports scheduled and event-driven tasks, and includes model routing and spending controls. Google is competing for the place where employees delegate their work, with its existing office suite providing a considerable head start.

For developers, EmbeddingGemma 2 maps text, code, images, audio, and video into one embedding space. The 740-million-parameter model ships under Apache 2.0, with modular encoders and on-device support. Searching a video collection with a voice memo becomes a practical local workflow.

RTX Spark and Microsoft Execution Containers put local agents at the center of the Windows pitch. MXC reaches general availability, providing OS-level infrastructure for persistent background agents. RTX Spark laptop preorders opened October 7, with availability October 16; compact desktops follow in November. Surface Laptop Ultra supports up to 128 GB of unified memory.

Microsoft also released Microsoft-Decision-1, which scores predefined answers in a single pass for routing, verification, and workflow control. Built by post-training Qwen3.5-9B, it is available through Foundry and OpenRouter. Microsoft reports 35× lower median latency than GPT-6 Sol in its tests. Even Microsoft’s decision model starts with Qwen.

Personal Agent Protocol, or Poppy, now has a published draft. Announced October 6 and released October 9, it defines how personal agents identify themselves, obtain customer-approved access, and work across a business’s website, APIs, and agents. OpenAI, Visa, PayPal, and others joined the design process. Built on existing standards including OAuth and MCP, Poppy addresses a concrete problem: businesses need to know whose agent has arrived and what it is allowed to do.

The company behind Jev raised an $870 million Series A at a $7.5 billion valuation, led by Andreessen Horowitz. TypeSafe plans more machine-native models and enterprise infrastructure. With Microsoft, Liquid, and Subquadratic entering the category, decision models have acquired competitors and a very substantial financing benchmark within weeks.

Open d1 brings decision models to local hardware. The released d1-3B accepts text and images; the experimental d1-omni-600M also supports text-and-audio inputs. Both produce structured answers in one forward pass without generating output tokens. Liquid reports 16-millisecond single-question latency for d1-3B on Jetson AGX Thor, with longer inputs taking more time. Weights and local inference support are available now.

Ai2’s Bolmo research appeared in Nature, accompanied by new checkpoints extending its method to Qwen and Llama. “Byteifying” converts existing subword models into byte-level models through additional training, avoiding a full retraining from scratch. The resulting Bwen and Blama models approach their source models’ performance while giving researchers another way to investigate the limitations of fixed token vocabularies.

October 5’s North 2 adds persistent memory, shared libraries, reusable skills, and application creation to Cohere’s enterprise platform. It supports different model providers and cloud, on-premises, or air-gapped deployment, alongside user quotas and organization-wide spending caps. Its appeal is the combination of agent capabilities with control over deployment and cost.

Mistral Large 4, officially nicknamed Le Chonk, enters public API preview. The natively multimodal mixture-of-experts model has one trillion parameters, with 52 billion active, and emphasizes coding, cybersecurity, finance, and law. Mistral says it trained the model on 3,800 Grace Blackwell GPUs in its own European datacenters. Downloadable weights are promised by month-end, so the open-weight announcement is ahead of the download button.

Arena raised $200 million at a $3.1 billion valuation and introduced an Alignment Index covering 27 models across 90,000 agent sessions. It measures three observable failures: unauthorized actions, false attribution, and claims of completion when work remains unfinished. These are useful, bounded measures, not a general certificate of safety.

Snorkel expanded its Open Benchmarks Grants to $30 million, adding benchmark red-teaming and a research fellowship. Checking whether the tests themselves can be gamed deserves funding too.

Announced October 12, SubQ Snap tackles the context problem inside decision models. It accepts up to 384K tokens of text and images, returning structured choices for tasks such as selecting tools, checking results, and deciding whether an agent should continue. Subquadratic reports processing 100K tokens in about a second and charges $0.05 per million input tokens, with no output charge. The interesting proposition is that a small decision may require a large amount of evidence, and discarding that evidence to save time can make the decision wrong.

🌟 RoboJEPA: Scaling Robotic Latent World Models (October 7) – Trains latent world models across 12 robotic embodiments, scaling to an 8B-parameter predictor. Prediction error follows a scaling relationship and correlates with downstream planning performance, including on physical robots. This gives researchers a potentially useful way to estimate improvements in control before running expensive hardware evaluations. The team releases checkpoints, training code, and deployment code. →read the paper

🌟 Long-WAM: Scaling the Context of World-Action Models (October 7) – Shows why adding visual history sometimes improves robot control and sometimes accomplishes little. With autoregressive pretraining, extending history to 19.2 seconds raises RoboCasa GR-1 success from 63.3% to 78.7%; a bidirectionally pretrained initialization gains nothing overall. Streaming and asynchronous execution make the approach practical for dynamic manipulation on real robots. →read the paper

VioLA: Learning Generalist Humanoid Control Policies from Human Data (October 8) – Predicts body and hand motion in shared latent spaces, which specialized controllers translate into robot movements. This lets human recordings supply 93.2% of its 140.6-million-frame training pool. The reported hardware results include locomotion and manipulation without task-specific fine-tuning, offering a concrete route around the scarcity of humanoid demonstrations. →read the paper

UNITAS: A 3D-Native World Action Model for Embodied Manipulation (October 8) – Places observations, actions, and scene changes in a shared metric 3D frame. Human hands and robot grippers become point trajectories, giving prediction and execution a common geometric representation. The 1.7B model reports improved scene prediction and 85% average success across its real-world tasks. Worth including for the representation itself and its connection to control. →read the paper

DreamTrue: Action-Faithful Robot World Model with Counterfactual Post-Training (October 8) – Addresses a consequential bias: models trained mostly on successful demonstrations can predict success for actions that should fail. DreamTrue modifies recorded actions to broaden training coverage and uses a reward model trained on human-annotated interaction defects. The human-assessed defect rate falls from 48.12% to 6.25% on AgiBot, providing evidence for better action-conditioned prediction. →read the paper

Kandinsky 6.0 Video: Foundation Models for Synchronized Video and Audio Generation (October 4) – Introduces 3B and 29B models generating five-second videos with synchronized audio, including speech and lip synchronization. A dual-stream architecture connects video and audio through bidirectional cross-attention. Code, checkpoints, and Diffusers integration are released under MIT, making this a substantial open multimedia release. Its demonstrated capabilities concern generation quality rather than physical planning. →read the paper

MiMo-V2.6: Scaling Reinforcement Learning Towards Self-Improvement (October 8) – An omni-modal model family whose report examines three dimensions of RL scaling: larger rollout batches, more diverse environments, and more computation devoted to grading. Particularly useful are the details on reward hacking, router stability, and coordinating different agent harnesses. The accompanying release covers training dynamics, RL environments, and the training framework. →read the paper

World-model research increasingly tests whether better predictions improve control, including whether models predict unsuccessful actions faithfully.

Shared motion and 3D representations make human recordings more useful for training robots.

Agents extract richer learning signals from completed trajectories, including attempts that receive identical rewards.

Memory research focuses on preserving decision-relevant evidence and testing whether compressed lessons retain it.

Code, replay, and physical measurements provide concrete checks on knowledge agents accumulate.

Compression and asynchronous execution make larger training runs and prediction-based control more practical.

Choosing what to investigate, and knowing when enough evidence has been gathered, remain major weaknesses in otherwise capable agents.

🌟 ARC: A Reasoning Recipe for Robot Foundation Models (October 8) – Adds reasoning traces that explain why the next action is appropriate and what it should change. These traces are generated from existing demonstrations, then used to adapt both vision-language-action and world-action models. The substantial gains make this important evidence that better supervision can improve physical decisions without collecting additional robot demonstrations. →read the paper

A Balanced Data Diet: Addressing the Exploration Bottleneck in Mega-Scale RL for Robot Control (October 8) – Finds that uniform sampling wastes increasing amounts of simulation on configurations a policy has already mastered or cannot yet solve. Success Guided Sampling concentrates training near the policy’s current capabilities. Experiments scale beyond one million parallel environments, with learned assembly policies subsequently transferred to physical hardware. A useful explanation of why more simulation does not automatically produce better learning. →read the paper

🌟 Memento 3: Model-Based Recursive Self-Improvement through Reflective Rulebooks (October 8) – An agent records hypotheses about an unfamiliar environment, compiles them into executable code, and checks predictions against observed transitions. Errors drive revisions while the underlying LLM remains frozen. It clears all 25 public ARC-AGI-3 games; the broader contribution is a concrete mechanism for making acquired world knowledge persistent, testable, and useful for planning. →read the paper

AgentGarten: Code Worlds for Evolving Agents (October 8) – Combines simulation backends that maintain explicit state and rules with a shared neural renderer that produces visual observations. Agents pass experience forward through reusable playbooks. The important contribution is an environment architecture that can expand through code while retaining consistent interactions and realistic visuals, creating more varied places for agents to practice. →read the paper

From Traces to Agentic Worlds: Agentic Language World Models for Interactive Environment Simulation (October 5) – Trace2Env builds a reusable “worldbook” from historical interaction logs, then uses an agent to simulate observations and persistent state changes. Across nine environments, actions developed in these simulations remain valid more often when replayed in the original systems. This matters when a training environment must be approximated from traces because the actual system is unavailable. →read the paper

🌟 Self-Retrospection Distillation: Turning Post-hoc Experiences into Prior Foresight (October 6) – Uses completed trajectories to teach a policy what it could have anticipated before acting. This supplies a learning signal when group-relative RL receives identical rewards across attempts, including when every attempt fails. Across ten tasks, it complements existing training methods; the foresight target is needed only during training, so it adds no corresponding generation requirement at inference. →read the paper

EVISKILL: Grounding Skill Evolution in Replayable Evidence (October 4) – Links proposed skill edits to the execution evidence and task conditions that justify them. Targeted replay tests individual corrections, while broader validation determines what enters the persistent skill. This addresses a practical weakness in agent learning: useful advice can become detached from the circumstances under which it was actually shown to work. →read the paper

REMORY: Learning Residual Memory for Context Compaction (October 8) – Supplements a textual summary with learned memory tokens that preserve information needed for subsequent decisions. On SummHay, it approaches the full-context joint score using 5.2% of the input positions. Agent evaluations also show fewer repeated tool outputs and errors, making it a substantive attempt to recover what ordinary summarization loses. →read the paper

Self-Generated Feedback Destabilizes Test-Time Training: A Causal Decomposition of Long-Horizon Adaptation (October 4) – Studies what happens when weight updates change the model producing its own future training text. Across long streams, the feedback loop worsens prediction on independent human-written text. Freezing the generator or checking updates against independent evidence reduces the damage. The findings concern the tested language-prediction settings, but identify a specific failure mechanism worth watching in continual adaptation. →read the paper

LoGRA: Scaling LLM Reinforcement Learning with Low-Rank Gradient Sketches (October 5) – Compresses gradient information and controls update size using predicted policy change. Reports up to 45.7% lower average training memory without sacrificing performance on the evaluated reasoning tasks. It also enables training a 27B model on an eight-GPU node where dense Adam runs out of memory. A meaningful addition because it changes which RL experiments fit on available hardware. →read the paper

Towards Looped Models Done Right, Part II: Rethinking at Fixed Points (October 5) – Uses the convergence of recurrent states to reduce training and inference costs. At 1.6B parameters, its learned depth prior matches the fixed-depth comparison’s downstream average with a three-times-smaller KV cache. The work connects repeated internal computation to practical savings in memory, prefill, and RL updates. →read the paper

RealtimeWAM: One-Step Asynchronous World Action Models (October 5) – Distills action generation into one step and overlaps computation between video and action components. Reports approximately 25× end-to-end speedup on H100 with less than 1% performance loss in its tested configurations. It earns a separate place for addressing two concrete deployment bottlenecks: repeated denoising and unnecessary waiting between model components. →read the paper

🌟 Optimizing the Optimizer: Language Models Discover Faster Molecular Relaxation Algorithms (October 5) – An agent rewrites a molecular geometry optimizer, with admission checks rejecting premature stopping and changes that fail on unseen molecules. The resulting AutoSella variants transfer to computational settings absent from the search. At the tested DFT level, the best variant uses 40.2–77.2% of Sella’s force evaluations for the same energy reduction. →read the paper

A Self-Learning Scientific Agent for X-Ray Diffraction (October 6) – Gan Jiang combines diffraction-analysis tools with executable skills that it revises and validates through experience. Skills are frozen before held-out evaluation, where they improve analytical performance across multiple tools and datasets. This provides a concrete scientific example of reusable expertise accumulating around a fixed language model and established physical models. →read the paper

🌟 RoboQuest: Generalist Physical Agents that Search, Inspect and Test (October 7) – Tests whether robots can seek missing information, inspect hidden properties, and investigate unfamiliar mechanisms. The best evaluated agent succeeds in only 23% of episodes, although supplying the hidden information lets agents perform most component actions. Failures frequently involve stopping exploration too early or failing to repair disturbances caused while investigating. →read the paper

Learn2Play Bench: How Well Do LLM Agents Learn from Experience in Unfamiliar Environments? (October 6) – Uses unfamiliar and counterintuitive game rules to separate learning through interaction from relying on prior knowledge. Complete action-and-feedback histories can outperform distilled rules or strategies, while changing the harness affects both performance and cost. This is an important counterweight to the assumption that converting experience into compact lessons reliably improves an agent. →read the paper