"Diffusion and flow policies buy multimodal action generation at the price of sampler and timing discipline."
A Grounded AI Agent
Figure 34.4 should be read as a decoding contract: denoising or flow steps, conditioning tokens, action horizon, control rate, and safety filter must fit inside the robot loop.
Build And Evaluation Checklist
Curriculum, depth, and self-containment. RDT-1B, pi-zero, FAST, and pi-zero point five show that action generation is now the central VLA design axis. For Diffusion and flow VLAs: RDT-1B, pi-zero, pi-zero FAST, pi-zero point five, the practical reading is to pin down the interface, assumptions, concrete example, and failure mode before comparing methods.
Production and evaluation contract. The durable comparison is action representation: diffusion, flow, or compressed autoregressive tokens. For Diffusion and flow VLAs: RDT-1B, pi-zero, pi-zero FAST, pi-zero point five, treat the diagram, code, table, exercise, warning, and references as one evidence packet: boundary, artifact, tool choice, transfer check, failure mode, and source grounding.
Before accepting a Diffusion and flow VLAs: RDT-1B, pi-zero, pi-zero FAST, pi-zero point five result, name the loop variable that changed, the tool that makes it reproducible, the failure that would fool the metric, and the source that backs the claim.
Write an evidence row for one diffusion or flow rollout: sampler steps, action horizon, control frequency, task metric, inference latency, and the failure label for late or unstable commands.
# Represent a short action chunk as a batch-ready tensor shape.
batch_size, horizon, action_dim = 32, 16, 7
def chunk_shape(batch_size: int, horizon: int, action_dim: int) -> dict[str, int]:
return {
"batch": batch_size,
"horizon": horizon,
"action_dim": action_dim,
"tokens_per_batch": batch_size * horizon,
}
print(chunk_shape(batch_size, horizon, action_dim))Use robomimic, Diffusion Policy, or LeRobot policy implementations to prototype action chunking before designing a new architecture. These libraries already manage temporal windows, normalization, batching, and rollout evaluation.
Diffusion and flow VLAs matter because some robot actions are better modeled as continuous trajectory distributions than as long symbol strings. These heads trade simpler decoding for richer motor expressivity on dexterous and high-rate tasks.
Why Continuous Action Heads Returned
Tokenizing action is attractive because it lets a VLA reuse language-model sequence machinery. The cost is that robot motion is continuous, high-frequency, and often multi-modal. A drawer can be pulled with slightly different wrist poses. A bimanual task can admit many coordinated trajectories. A single discrete next token can be too brittle for this geometry.
Diffusion and flow action heads address this by generating action chunks as continuous trajectories. Diffusion policies learn to denoise action sequences conditioned on observations. Flow matching learns a vector field that transports noise into actions. Both routes let the model represent multiple plausible futures without forcing every motor detail through a small set of bins.
RDT-1B and pi-zero are best read as VLA systems whose action head is a trajectory generator. The vision-language backbone supplies context, while diffusion or flow supplies smooth continuous control.
RDT-1B, pi-zero, pi-zero FAST, and pi-zero point five
RDT-1B scales a diffusion transformer for bimanual manipulation and predicts action chunks from language plus multi-view RGB inputs. Pi-zero uses a flow-matching head built on a pretrained VLM to generate continuous control for diverse robots. Pi-zero FAST revisits autoregressive action generation by improving tokenization with frequency-space compression. Pi-zero point five adds heterogeneous co-training to push open-world generalization on mobile manipulation tasks.
These systems should not be collapsed into one category. RDT emphasizes bimanual diffusion at scale. Pi-zero emphasizes flow matching for general robot control. FAST emphasizes efficient action tokenization. Pi-zero point five emphasizes co-training across diverse sources for more robust generalization.
Imagine sampling random action noise and learning a velocity field that moves it toward demonstrated action chunks. At inference time, the model follows that learned field for a small number of steps. The result is a continuous action sequence conditioned on the current scene and instruction.
The mathematical sketch is compact: learn $v_\theta(x_t, t, c)$ so that samples move from a noise distribution toward demonstrated action chunks under context $c$. The context includes images, language, and robot state. The practical question is how many flow or denoising steps the controller can afford before latency breaks the loop.
Use diffusion or flow heads when the task needs smooth multi-step motor behavior, multiple plausible action modes, or dexterous contact. Use tokenized autoregression when discrete sequence modeling, fast sampling, or language-model compatibility dominates. Revisit the choice after measuring latency and closed-loop recovery, not before.
A beautiful action distribution is not useful if inference misses the control deadline. Always report action horizon, inference time, control frequency, and whether the controller can reuse an action chunk while the next chunk is generated.
Treat diffusion and flow vlas: rdt-1b, pi-zero, pi-zero fast, pi-zero point five like a control-room label. If the label does not tell a future debugger what moved, what sensed, or what failed, it is decoration rather than engineering knowledge.
The active frontier is hybridization. FAST shows that better tokenization can make autoregressive VLAs competitive on high-frequency actions, while flow and diffusion systems keep improving sampling speed. The durable lesson is that action representation is an algorithmic choice, not a naming convention.
Expected output: Diffusion and flow VLAs: RDT-1B, pi-zero, pi-zero FAST, pi-zero point five should leave a reproducible VLA evidence trace with checkpoint, action representation, robot interface, metric, and failure label.
Why might a bimanual manipulation task benefit from a diffusion or flow action head? Your answer should mention multi-modality, action chunks, and latency.
Diffusion and flow heads are VLA action generators for continuous, multi-modal control. They trade sampling complexity for smoother and richer action distributions.
For each task, choose tokenized autoregression, diffusion, or flow: pushing a block, folding cloth, opening a drawer, and mobile pick-and-place. Give one reason and one evaluation metric for each choice.
What's Next?
Section 34.5 zooms in on action representation, including the FAST tokenizer.
Liu et al. (2024). "RDT-1B: a Diffusion Foundation Model for Bimanual Manipulation." arXiv.
RDT-1B studies diffusion transformers for language-conditioned bimanual manipulation at large scale. It is especially relevant for readers comparing tokenized autoregression with continuous denoising heads.
pi-zero uses a flow-matching action head on top of a pretrained vision-language backbone. The paper is central for understanding why continuous action generation became a serious alternative to discretized action tokens.
FAST uses frequency-space compression to tokenize continuous action sequences for autoregressive VLAs. It is the key source for the chapter distinction between naive per-dimension binning and compressed action-sequence tokenization.
Pi-zero point five extends pi-zero through heterogeneous co-training for broader open-world generalization. It is useful for readers studying the frontier between task-specific robot policies and household-scale generalist behavior.
Chi et al. (2023). "Diffusion Policy: Visuomotor Policy Learning via Action Diffusion." arXiv.
Diffusion Policy established denoising over action sequences as a strong imitation-learning recipe. It gives the mathematical and practical background for diffusion heads in later VLA systems.
Bjorck et al. (2025). "GR00T N1: An Open Foundation Model for Generalist Humanoid Robots." arXiv.
GR00T N1 frames humanoid control as a dual-system VLA architecture with reasoning and fast action generation. It prepares the transition from Chapter 34 into Chapter 35 and the later humanoid chapter.