Section 35.1: Why foundation models matter for robotics

"A robot prior earns the word foundation only when it reduces the cost of the next real adaptation."

A Practical Robot Theorist

The Problem Foundation Models Solve

Training a separate robot policy for every robot, sensor stack, and task wastes data. Many embodiments still need the same subskills: localizing objects, recognizing task language, predicting contact-rich motion, and recovering from small perturbations. A foundation model tries to absorb those invariants once, then expose them through a shared latent space or policy backbone.

The motivating failure mode is familiar from earlier imitation-learning chapters: a narrow policy can look excellent on the one setup it saw during data collection and collapse as soon as the camera moves, the gripper changes, or the object geometry shifts. A foundation model is an attempt to turn those brittle one-off policies into an adaptation problem rather than a full retraining problem.

Formal View: Pretrain Once, Adapt Many Times

A useful abstraction separates the shared backbone from the robot-specific adaptation layer:

$$\theta^*=\arg\min_\theta \sum_{r \in \mathcal{R}} \lambda_r \; \mathbb{E}_{\tau \sim D_r}[\ell(f_\theta, \tau)], \qquad \phi_{r'}^*=\arg\min_\phi \; \mathbb{E}_{\tau \sim D_{r'}}[\ell(f_{\theta^*,\phi}, \tau)] + \lambda \lVert \phi \rVert_2^2.$$

The first stage learns a shared prior across training robots $\mathcal{R}$. The second stage adapts that prior to a new robot $r'$. The adaptation term $\phi$ might be a LoRA block, an action adapter, an embodiment token, or a small amount of post-training data. The operational question is whether the sample count and wall-clock needed for adaptation drop enough to matter in practice.

Code Fragment 1 turns that idea into a numeric transfer audit. The point is not the arithmetic itself. The point is to make the "foundation" claim observable as a reduction in adaptation data.

# Compare scratch training against adaptation from a shared prior.
# The adaptation gain is the ratio that matters, not the absolute count alone.
robots = {
    "tabletop_arm": {"scratch_demos": 1800, "adapt_demos": 220},
    "mobile_manipulator": {"scratch_demos": 2600, "adapt_demos": 410},
    "bimanual_platform": {"scratch_demos": 5200, "adapt_demos": 900},
}

for name, stats in robots.items():
    gain = stats["scratch_demos"] / stats["adapt_demos"]
    print(f"{name}: adaptation_gain={gain:.1f}x")

tabletop_arm: adaptation_gain=8.2x
mobile_manipulator: adaptation_gain=6.3x
bimanual_platform: adaptation_gain=5.8x

The expected output is a transfer audit where every embodiment shows an adaptation gain comfortably above 1x. These values suggest that the shared prior is buying real sample-efficiency, not merely shifting where the same amount of tuning work is paid.

What Gets Shared, What Stays Local

The table above is the central systems lesson. A robot foundation model does not erase embodiment. It changes where embodiment enters the stack.