VLA Models Explained: Architecture, Types & the Leap to VLA+

One of the most talked-about VLAs comes from Google DeepMind, and their bet was on reasoning itself. They started with Gemini 2.0 – a VLM that could see, describe, and reason about images, including spatial and physical relationships, but couldn't act.

With additional training focused on embodied reasoning, they extended it into Gemini Robotics-ER. This version can reason about the physical structure of the world and understand how actions would play out. It understands where objects are in space, infers relationships like distance, orientation, and containment, tracks objects across multiple camera views, predicts simple motion paths based on task descriptions, and reasons about how an action would change the scene.

This already makes the Gemini model applicable for basic robot control, without any robot-specific training. It does this through zero-shot control by generating code that uses perception and control APIs, and few-shot control by imitating a small number of example trajectories provided in the prompt.

The key insight from Google here is that embodied reasoning can emerge inside general multimodal models like Gemini 2.0, and that this reasoning can be reused across tasks, environments, and robots. But Gemini 2.0 itself is not a full robotics system.

As the next step, Gemini Robotics appeared, built on top of Gemini Robotics-ER, turning its reasoning ability into a full VLA model that takes images of the scene plus a text instruction and turns them directly into robot actions.

From such models, we need minimal latency and maximum efficiency for physical control, while still benefiting from a large, general-purpose model. DeepMind split Google Gemini into two parts for this:

Out of the box, Gemini Robotics can solve many short-horizon dexterous tasks, such as opening containers and drawers, pouring and placing objects, manipulating deformable items like cloth or cables, and coordinating two hands in cluttered environments.

It also follows natural-language instructions, even in scenes with unseen objects or new layouts, and Gemini Robotics bets on generalization. But it doesn't fully solve robotics. Long-horizon planning and extreme dexterity are still challenging. In any case, this is Google DeepMind's important statement in foundational physical AI.

While Gemini Robotics turns large multimodal models into general robot policies through embodied reasoning extended with a cloud-to-robot architecture, Physical Intelligence proposed a different path entirely – one that treats action generation as a first-class modeling problem.

They take a pre-trained VLM (in their case, PaliGemma) and pair it more tightly with continuous action modeling via a dedicated flow-matching action expert to create π0 – a generalist robot control model.

Its separate action expert module is trained jointly with a single transformer backbone for both perception and action. Training happens on many robot configurations like single-arm and dual-arm robots and mobile manipulators, with different action spaces (this is also called cross-embodiment training) to achieve the needed level of generalization.

So, π0's main difference from Gemini Robotics is that it directly encodes continuous motion distributions rather than relying on discrete decoding or planning modules. Thanks to this, and by using a diffusion-style mechanism, it predicts chunks of future actions, like a 50-step action sequence.

This matters because robots move continuously through space, and predicting trajectories rather than single timesteps lets the model capture smooth, coordinated motion. It's the difference between telling a robot "move your arm here" versus "here's the path your arm should follow."

Most VLAs like Gemini Robotics and π0 are really large models, increasing their capabilities with scaling. But how small can we make a VLA without breaking usefulness?

Hugging Face found the answer, and it involves rethinking what "necessary" means. They're working on making AI technologies accessible for everyone, through open source and by leaning into "smallness" and practicality of the models. Their VLA is no exception.

SmolVLA starts from a compact pretrained vision-language model, and to make it even lighter, the last layers of the VLM are removed, only a small number of visual tokens are kept, and the remaining layers embed three inputs: the language instruction, one or more RGB images, and the robot's sensorimotor state.

These combined representations are then passed to a small action expert responsible for producing robot actions. The architecture of SmolVLA and its working style is similar to π0, except for one special feature – it uses an asynchronous inference setup to make very fast responses. Perception and action prediction run separately from action execution. While the robot is executing one chunk of actions, the model can already process the next observation and predict future actions. That's why control rates stay high even on limited hardware.

There are many advantages to pursuing efficiency instead of scale:

So SmolVLA indeed lowers the barrier to entry, making it possible for more researchers, labs, and hobbyists to experiment with VLA models using open code, open data, and affordable robots.

While some models aim for universality, others narrow their specialization to a particular type of embodied AI – and sometimes that focus unlocks capabilities that general models can't reach.

Figure, the creators of Figure 03 robot, developed a VLA specifically for humanoid robots with a focus on controlling the entire upper body in real time. They followed an idea: general reasoning and fast control do not have to live in the same model, but they must be tightly connected.

Their VLA is called Helix and it turns language understanding directly into coordinated humanoid motion to master the control of robots that have many joints, move continuously, and operate in messy environments like homes.

Helix uses a two-system architecture, inspired by the idea that different parts of intelligence operate at different timescales:

Using this separate architecture, Helix can "think slowly" about goals but still react quickly to physical changes. Unfortunately, Helix still depends on high-quality teleoperation data and lacks explicit long-term world models, but it is the reason why Figure 03 robots can pick up thousands of unseen household objects using only language, perform long-horizon tasks without predefined scripts, and work in collaborations of two robots with the same model weights, adapting motions in real time.

The next approach makes an interesting addition to the classic VLA architecture from a different angle – what if the VLM backbone itself needs restructuring?

All variants of VLAs we've discussed share one general feature – they're based on VLMs as a straightforward way to obtain the vision and language understanding so needed for VLAs. However, researchers from Midea Group and East China Normal University noticed that after training specifically for robot control, VLMs tend to forget part of their original knowledge.

Sometimes this backfires on simple tasks, like solving a basic equation written on a whiteboard, just because that exact case never appeared in the robot training data. It appears that VLM general capabilities and the new action component in VLAs may compete for the same parameters, which is the very cause of knowledge loss during fine-tuning.

That's why the researchers created their own VLA, called ChatVLA-2, which uses a Mixture-of-Experts (MoE) design inside the VLM backbone:

This MoE architecture prevents hard competition for parameters and, at the same time, the model keeps shared representations where reasoning and action need to connect!

ChatVLA-2 also adds a lightweight reasoning-alignment mechanism to ensure that the action expert's outputs are consistent with the model's internal reasoning and don't drift to mimicking demonstrations blindly. The secret is in two-stage training:

The second stage encourages the action module to rely on preserved reasoning and not overwrite it when it is already correct.

With ChatVLA-2 as a "brain" preserving fundamental VLM knowledge, robots can do things like read and solve a math problem from a whiteboard and pick the right card, or follow new spatial instructions to place unfamiliar objects correctly.

As a result, ChatVLA-2's approach focuses on preserving correct reasoning and spreading its effect on the action part. But there are also some uncovered secrets in the action space itself.

A recent approach from Beihang University and AgiBot argues that many previous methods reason in the wrong space. If the goal is better actions, then reasoning itself should happen in action space.

For this, the researchers replace language or visual chain-of-thought with a chain of action-level reasoning, which they called Action Chain-of-Thought (ACoT).

Guidance like "move to object → grasp → lift" (language), or "this is what the goal image looks like" (vision), is too semantic and abstract for robot actions that are continuous, high-dimensional, and kinematic. ACoT brings back that kinematic meaning with coarse action intents, such as approximate trajectories or motion patterns, that already live in the same space as the final control commands.

ACoT-VLA has two complementary components:

So while EAR provides explicit motion guidance, IAR gives implicit behavioral bias. This is how they form the Action Chain-of-Thought, which conditions the final action policy and guides low-level control in a grounded way.

But what if we don't add any of these additional architectural tricks to VLMs at all?

NVIDIA researchers explored an intriguing idea: what's the minimal way to turn a VLM into a robot controller? The key design choice is what not to change – no new tokens, no new layers, no changes to the vocabulary.

This VLA-0 model represents robot actions as plain text and generates them with its normal text output. Here's how it works:

Resolution can be increased just by changing the numeric range, without touching the model.

VLA-0 also includes an interesting training feature: Masked Action Augmentation randomly hides parts of the target action sequence during training, so the model won't simply continue numeric patterns mechanically. Just as intended, VLA-0 keeps relying on the image and instruction to infer actions, improving robustness and reducing overfitting.

To prove that this straightforward approach works, here are the results:

Sometimes the simplest path forward is the one that changes the least.

And now, with this landscape mapped, we can return to where we started: Microsoft's leap to VLA+.

Just today, Microsoft introduced its first robotics model derived from their famous Phi family of vision-language models. Еhey called it Rho-alpha (ρα) and it targets bimanual manipulation under natural-language control. It takes language instructions and turns them into low-level control signals for dual-arm robotic systems.

The main thing is that this model is positioned as a step toward a VLA+ model. Let's see why.

Rho-alpha extends perception and learning beyond what most VLA systems support today.

Both these sides make adaptability central rather than incidental. With tactile sensing, the model can detect mismatches between intended and actual outcomes during interaction, plus update its behavior while deployed. Adaptation to errors and changing conditions becomes a primary expression of intelligence and not something that is fixed after the fact.

We should remember though that one of the biggest bottlenecks in robotics is data. If we talk about tactile sensing, there is no web-scale corpus of such tactile embodied interaction.

Rho-alpha addresses this problem through a hybrid training pipeline:

Simulation plays a central role here. Microsoft Research uses NVIDIA Isaac Sim to generate physically accurate synthetic data, including interactions that are difficult or impractical to collect via teleoperation alone. This simulated data is combined with real-world robot datasets (commercial and open) to make up for the lack of tactile data.

So, what can this model do today?

In demonstrations, Rho-alpha controls dual-arm robot setups to perform tasks such as:

So, Rho-alpha is currently being evaluated on dual-arm systems and humanoid robots, with ongoing work focused on end-to-end optimization of the training pipeline and data efficiency. This is a part of Microsoft’s longer-term push toward foundational physical AI, and Rho-alpha is a scalable blueprint for adaptable, learning physical systems with a more extended loop, where language, perception, touch, and human feedback stay all together.

A full technical description is expected in the coming months, and we hope there will be much more to explore then.

Before VLAs split into flow matching, action experts, MoE backbones, and VLA+, two models helped establish the basic recipe everyone is now trying to improve to build novel VLAs.

RT-2 tested if robot actions could be treated almost like another language. Google DeepMind took vision-language models trained on web data and taught them to output robot actions as tokens. Suddenly, knowledge learned from images and text could affect what a robot actually did in the physical world. You can still see this idea in newer approaches like VLA-0, only pushed much further.

Then OpenVLA opened that recipe up. The 7B model was trained on 970K robot trajectories from Open X-Embodiment and released with weights, code, and support for adapting it to new robots. That’s why you can download its weights, run inference yourself, and fine-tune it for a new robot with methods including LoRA.

After eight new and two classical architectures, you may come to the obvious question: which of these can you actually open on your computer and experiment with?

SmolVLA is probably the easiest place to start. It is only 450M parameters, lives inside Hugging Face's LeRobot ecosystem, and was deliberately designed to bring VLA experiments down from giant robotics labs to more ordinary hardware.

First, install LeRobot with SmolVLA support:

Then load the pretrained policy:

This loads the model locally. To actually run inference, you'll need to provide observations such as camera images, the robot's current state, and optionally a natural-language instruction; SmolVLA then outputs continuous robot actions.

If you want to adapt it to your own robot data, LeRobot provides a training command:

As for the hardware, SmolVLA was designed to make VLA experimentation possible on relatively accessible hardware. Hugging Face reports that a 20,000-step run takes about four hours on a single A100 GPU – that is for fine-tuning. Actual requirements depend heavily on the robot, image inputs, batch size, and whether you're running inference or training.

VLA models are a major step toward Physical AI, but (as it always happens) they are not a complete solution for robotics yet. The approach works best when perception, language, and action can be connected through enough data and a relatively clear task structure. It becomes much harder in messy, long-horizon, safety-critical, or highly dexterous environments.

In this episode, we've walked through a landscape of VLA models, each presenting their own path from VLM perception, understanding, and reasoning to what robots can execute in the real world. To make it clear how to choose a model or an idea for building a VLA model for your purpose, here is a summary of the exclusive features:

VLAs help robots do things, acting as an "interface" between perception, language, and action, and you can define the rules of how this interface works. But VLAs don't fully solve robotics – there are many more pieces needed to complete the puzzle: world models, memory systems, long-horizon planning, data engines, prediction, and safety. The real challenge now is making all these components work together for the upcoming age of Physical AI.