Section 41.5: Risks of generated experience

The open problem is not whether diffusion can plan; it is whether it can plan fast enough to close a 100 Hz loop, and trustworthily enough that its generated futures do not lie.

A Frontier Of Fast, Honest Generation

Two frontiers decide whether diffusion and flow models graduate from impressive demos to deployed controllers: speed and trust. Speed, because a control loop cannot wait for a thousand denoising steps. Trust, because generated futures and generated experience can be confidently wrong, and a planner that optimizes against a fabricated future inherits the fabrication.

This section surveys the speed frontier (faster samplers and structural priors like equivariance), then gives a risk vocabulary for generated experience: when to reject synthetic rollouts, when to down-weight them, and when to treat them only as proposal data for a stricter downstream verifier.

Speed: From a Thousand Steps to a Handful

Vanilla DDPM sampling runs the full reverse chain, on the order of 1000 steps, which is far too slow for control. DDIM reinterprets the trained model as a deterministic ODE and skips steps, bringing sampling down to roughly 10 to 50 steps with little quality loss. Consistency models go further, distilling the generative process so that 1 to 4 steps suffice, which is the regime that makes diffusion-style control plausible at high rates. In practice these samplers move continuous-control diffusion from offline planning into the loop, though most deployed systems still run near 10 to 20 Hz rather than the 100+ Hz that stiff contact-rich control wants.

Equivariance: Building in the Symmetry of Rigid Bodies

Rigid-body actions live in SE(3), the group of 3D rotations and translations, and a grasp that is correct under one object pose is correct under a rotated pose too. SE(3)-equivariant diffusion bakes this symmetry into the network so the generator does not have to learn it from data: rotate the scene, and the generated action rotates with it. The payoff is sharply better sample efficiency and out-of-distribution pose generalization for manipulation, because the model spends its capacity on the task rather than on relearning geometry.

Theory

A simple way to express support mismatch is to compare the planner's training distribution $p_g(\tau)$ with the real deployment distribution $p_r(\tau)$. If high-scoring generated trajectories live where $p_r(\tau)$ is small, then optimizing on them can improve offline metrics while harming deployment.

One practical audit statistic is the weighted divergence

$$ \Delta = \mathbb{E}_{\tau \sim p_g}\left[w(\tau)\, \mathbf{1}\{p_r(\tau) < \epsilon\}\right], $$

where $w(\tau)$ is the training weight. Large $\Delta$ means the learner is spending too much attention on low-support generated experience.

Worked Example

The probe below performs a toy support audit. It marks generated plans that would receive high training weight even though their estimated real support is low.

# Refine noisy action trajectories toward a goal-conditioned plan.
import numpy as np

real_support = np.array([0.82, 0.63, 0.11, 0.07], dtype=np.float32)
train_weight = np.array([0.30, 0.25, 0.25, 0.20], dtype=np.float32)
epsilon = 0.15

low_support_mass = float(train_weight[real_support < epsilon].sum())
print({
    "low_support_mass": round(low_support_mass, 2),
    "should_reweight": low_support_mass > 0.3,
})

The expected output should raise the reweight flag. The interpretation is simple: too much of the training objective is being spent on trajectories with weak real support, so the synthetic mix is no longer trustworthy as-is.

Practical Recipe

1. Write the observation, action, horizon, and success metric before choosing a model.
2. Build a baseline that is simple enough to debug by inspection.
3. Add the maintained implementation only after the baseline behavior is understood.
4. Save one artifact containing configuration, seed panel, traces, metrics, and failure labels.
5. Run at least one perturbation test before trusting the result.

The safest production pattern is tiered trust. Let generated scenes help with representation learning first, let generated trajectories help with planner proposal generation second, and reserve direct control-policy training on synthetic experience for cases where support audits and hardware shadow tests stay clean.

That tiering also gives a clean course-design story. Students learn that "more data" is not a universal good; data must be scored by realism, support, and downstream consequence.

For Risks of generated experience, keep one inspectable probe for the model assumption, then use maintained libraries without changing the artifact schema used for baseline comparison.

1. Write the observation, action, state estimate, success metric, and rejection criterion.
2. Run a deterministic smoke test on one seed and save the complete configuration.
3. Add one perturbation tied to the section topic: delay, noise, horizon length, contact change, distractor object, or generated-scene shift.
4. Compare only methods evaluated by the same script, split, seed panel, and metric definition.
5. Record a postmortem that assigns failures to perception, representation, dynamics, planning, control, data coverage, timing, or evaluation.

When Risks of generated experience fails, do not collapse the result into a single method verdict. Assign the failure to the interface that broke, rerun one controlled perturbation, and keep the trace next to the metric. That habit turns a disappointing rollout into a reusable diagnostic asset.