Section 41.3: Generative trajectory planning and scoring

Stop learning to reverse a thousand noising steps; learn the straight-line velocity that carries noise to an action in one smooth flow.

A Flow-Matching Action Expert

Diffusion learns to reverse a noising process step by step. Flow matching learns something simpler and often faster: a velocity field that transports a noise sample to a data sample along a smooth path, sampled by integrating an ordinary differential equation (ODE). For action generation this matters because the ODE can be integrated in a handful of steps, and the training target is a clean regression on velocity rather than on noise across a long schedule.

This section connects flow matching for action generation to the scoring problem that any generative planner must still solve. A flow-matching expert proposes actions cheaply; receding-horizon execution then scores and filters those proposals at every observation refresh rather than once at episode start.

Theory

Flow matching defines a time-indexed path $x_t$ for $t\in[0,1]$ that starts at a noise sample $x_0\sim\mathcal{N}(0,I)$ and ends at a data sample $x_1$ (an action or action chunk). A learned velocity field $v(x_t, t, c)$ generates the sample by integrating the ODE

$$ dx_t = v(x_t, t, c)\,dt, $$

from $t=0$ (noise) to $t=1$ (action), conditioned on context $c$. Conditional Flow Matching chooses the simplest possible path: a straight line between a paired noise sample and data sample, $x_t = (1-t)\,x_0 + t\,x_1$. Differentiating gives a constant target velocity $x_1 - x_0$, so the training objective is the regression

$$ \mathcal{L}_{\text{CFM}} = \mathbb{E}_{t, x_0, x_1}\left[\left\lVert v_\theta(x_t, t, c) - (x_1 - x_0)\right\rVert^2\right]. $$

This is markedly simpler than DDPM training: no noise schedule to tune, no variance terms, and straight-line paths that integrate accurately in few Euler steps. Once an expert proposes actions this cheaply, the planner still faces the scoring problem. For $k$ candidate trajectories a practical scoring rule is $S(\tau^{(k)}) = \lambda_r R - \lambda_c C - \lambda_d D - \lambda_\ell L$, combining task reward $R$, collision or constraint penalty $C$, dynamics mismatch $D$, and latency or effort $L$; the planner keeps the highest-scoring candidate that also passes a hard feasibility filter.

Worked Example

The probe below shows a flow-matching forward pass and a few-step ODE integration. We interpolate linearly between a noise sample $x_0$ and a target action $x_1$, read off the constant target velocity $x_1 - x_0$, then integrate the ODE from noise to action with a handful of Euler steps. In a trained system the constant velocity is replaced by the learned field $v_\theta(x_t, t, c)$, but the path geometry and the integration loop are identical.

# Conditional flow matching: straight-line path from noise to action.
# x_t = (1 - t) x0 + t x1 ;  target velocity v = x1 - x0  (constant along the line).
import numpy as np

rng = np.random.default_rng(2)
x1 = np.array([1.0, 0.0])               # target action (data sample)
x0 = rng.normal(size=2)                 # noise sample

print("forward path and target velocity:")
for t in np.linspace(0.0, 1.0, 6):
    x_t = (1.0 - t) * x0 + t * x1       # point on the path at time t
    v = x1 - x0                         # CFM regression target
    print(f"  t={t:.1f}  x_t={x_t.round(3).tolist()}  v={v.round(3).tolist()}")

# Inference: integrate dx = v dt from t=0 (noise) to t=1 (action) with Euler steps.
x = x0.copy()
dt = 0.2
for _ in range(5):
    v_theta = x1 - x0                   # a trained v_theta(x, t, c) would go here
    x = x + v_theta * dt
print("integrated endpoint:", x.round(3).tolist(), " target:", x1.tolist())

The velocity is constant along the straight path, which is the whole appeal: the network only has to regress a stable target, and five Euler steps reach the action exactly. Replace the constant velocity with $v_\theta(x_t, t, c)$ and the same loop becomes a real action sampler. The proposed action still enters the scoring stack, so raw proximity to a target is never the final word: an embodied planner only commits once feasibility and effort penalties agree.

Practical Recipe

1. Write the observation, action, horizon, and success metric before choosing a model.
2. Build a baseline that is simple enough to debug by inspection.
3. Add the maintained implementation only after the baseline behavior is understood.
4. Save one artifact containing configuration, seed panel, traces, metrics, and failure labels.
5. Run at least one perturbation test before trusting the result.

The right mental model is proposal plus filter. Let the generator provide diversity, then use fast learned scorers and hard geometric checks to remove plans that are dynamically or spatially impossible. This usually beats asking the diffusion model alone to internalize every safety rule.

That is also the cleanest teaching pattern. Students can inspect the sampled plans, the scorer output, and the feasibility filter separately, which makes failure attribution far less mysterious than treating the planner as a single opaque block.

For Generative trajectory planning and scoring, keep one inspectable probe for the model assumption, then use maintained libraries without changing the artifact schema used for baseline comparison.

1. Write the observation, action, state estimate, success metric, and rejection criterion.
2. Run a deterministic smoke test on one seed and save the complete configuration.
3. Add one perturbation tied to the section topic: delay, noise, horizon length, contact change, distractor object, or generated-scene shift.
4. Compare only methods evaluated by the same script, split, seed panel, and metric definition.
5. Record a postmortem that assigns failures to perception, representation, dynamics, planning, control, data coverage, timing, or evaluation.

When Generative trajectory planning and scoring fails, do not collapse the result into a single method verdict. Assign the failure to the interface that broke, rerun one controlled perturbation, and keep the trace next to the metric. That habit turns a disappointing rollout into a reusable diagnostic asset.