Section 35.6: Data scale, compute, and the open-vs-closed divide

"Every scaling curve is secretly a budget memo with philosophical opinions about openness."

A Research Lead With Spreadsheets

Why Scaling In Robotics Is Different

Language models can often scale by adding text and compute while keeping the evaluation channel cheap. Robotics is more stubborn. Each extra demonstration may require hardware time, operators, resets, safety review, and embodiment-specific calibration. A compute-heavy training run can still be bottlenecked by the cost of generating trustworthy robot data and validating it on real platforms.

That is why open and closed stacks create different research tempos. Closed systems may report impressive results with inaccessible data or infrastructure. Open stacks may lag in headline numbers while moving faster in community debugging, fine-tuning, and independent validation.

A Simple Scaling Lens

A stylized way to think about the trade-off is

$$P \approx f(D, C, E),$$

where $D$ is data diversity and volume, $C$ is compute for training and serving, and $E$ is evaluation throughput on trustworthy scenario panels. Many labs can buy more compute faster than they can buy more credible robot evidence, which means progress saturates on the least glamorous axis.

A more operational view is to treat research throughput as

$$T_{\mathrm{iteration}} \approx \max(T_{\mathrm{data\ prep}}, T_{\mathrm{training}}, T_{\mathrm{hardware\ evaluation}}) + T_{\mathrm{failure\ analysis}}.$$

This decomposition makes the open-versus-closed divide more concrete. A closed stack may lower apparent model-development time because the policy arrives pre-integrated, but it often raises attribution cost because the lab cannot inspect which part of the performance came from data curation, architecture, post-training, teleoperation quality, or evaluation filtering. An open stack may start from a weaker absolute capability level while still producing faster scientific learning because failures are easier to localize and adaptation loops are easier to rerun.

Code Fragment 1 turns this intuition into a toy budget calculation.

# Compare where the budget goes in two robot-foundation-model programs.
programs = {
    "open_lab": {"data_hours": 400, "gpu_days": 120, "real_eval_days": 40},
    "closed_frontier": {"data_hours": 5000, "gpu_days": 900, "real_eval_days": 120},
}

for name, vals in programs.items():
    evidence_pressure = vals["real_eval_days"] / vals["gpu_days"]
    print(f"{name}: evidence_pressure={evidence_pressure:.2f}")

open_lab: evidence_pressure=0.33
closed_frontier: evidence_pressure=0.13

The expected output is a higher evidence-pressure ratio for the smaller open program, meaning a larger fraction of its iteration budget is spent on real-world validation instead of pure training throughput. That does not automatically make the open path worse; it often means the lab is paying for more inspectable evidence per unit of model development.

Mechanisms Behind The Cost Curve

Data scale matters in at least three different ways: the number of embodiments represented, the diversity of tasks and scenes, and the quality of state-action alignment inside each trajectory. Compute interacts with those axes asymmetrically. More FLOPs can help fit larger mixtures or more expressive action models, but they cannot repair missing calibration metadata, poor reset discipline, or under-specified action semantics.

This is where the open and closed worlds separate mechanistically. Open stacks usually expose dataset schema, action conventions, and training code, so their common failure mode is limited scale or uneven embodiment coverage. Closed stacks may demonstrate stronger integrated performance, but their common scientific weakness is that the source of the gain becomes entangled with private data mixtures, private evaluation filters, and hidden post-training procedures.

Open Versus Closed Is A Research Trade-Off

Evaluation Consequences

The choice of stack changes what kind of science a lab can do. Closed systems are often useful as frontier references or upper-bound demonstrations, but they are weak substrates for careful ablations because too many causal variables are hidden. Open systems are usually the right substrate for adaptation studies, mixture design, action-interface debugging, and course-ready assignments because every assumption can be written into a manifest and challenged.

The minimum evidence bundle for this section should include one construct-matched task panel, embodiment labels, dataset provenance, training or fine-tuning config, latency notes, and a failure taxonomy saved in the same artifact. Without that bundle, a stronger demo may still be a weaker scientific claim.