Section 24.5: Curating and mixing data

"More data helped until the easy task brought all its friends."

A Mixture Weight Optimizer

Mixture Weights

Let dataset sources be $D_1,\ldots,D_K$ with sampling weights $p_1,\ldots,p_K$. Training samples source $k$ with probability $p_k$, then samples an episode from that source. Uniform-over-episodes favors large datasets; uniform-over-sources favors small datasets; temperature sampling interpolates between them:

$$p_k = n_k^{ au} / \sum_j n_j^{ au},$$

where $n_k$ is source size and $ au$ controls how strongly size influences sampling. $ au = 1$ is proportional to size, while $ au = 0$ is uniform over sources.

Code Fragment 1 computes temperature-sampled mixture weights. This is the smallest practical tool for making a mixing policy explicit.

# Compute source-sampling weights for a mixed robot dataset.
# Temperature tau controls whether large sources dominate the training stream.
sources = {"OpenX": 1_000_000, "DROID": 76_000, "BridgeDataV2": 60_096}
tau = 0.5
raw = {name: count ** tau for name, count in sources.items()}
total = sum(raw.values())
weights = {name: round(value / total, 3) for name, value in raw.items()}
print(weights)

The expected output shows a middle ground: OpenX remains the largest source, but DROID and BridgeData V2 receive enough probability to affect training. If $\tau$ were 1, raw size would dominate more strongly; if $\tau$ were 0, every source would receive equal probability regardless of size. The correct value depends on the deployment claim and should be chosen before looking at final evaluation scores.

Bias And Coverage Audits

Mechanism: Mixtures Change Gradient Pressure

During training, a source with higher sampling probability contributes more gradient updates. That means the mixture determines which visual backgrounds, robot bodies, task verbs, and action ranges the model practices most often. If one large source contains mostly easy pick-and-place episodes, the model can become excellent at those motions while under-practicing rare tool-use or recovery behaviors.

A good mixing manifest therefore records both source weights and batch composition rules. Some teams use per-source batches so every update sees multiple embodiments. Others use task-balanced sampling so rare skills are not drowned out. Either choice is defensible when the manifest makes it reproducible and the evaluation reports per-source outcomes.

Failure Analysis For Data Mixtures

When a mixed-data policy fails, the first question is whether the failure came from lack of coverage, negative transfer, or source conflict. Lack of coverage means the deployment condition barely appears in any source. Negative transfer means another source teaches a behavior that is actively wrong for the target robot or task. Source conflict means two datasets use similar observations or instructions but incompatible action semantics, success definitions, or reset distributions.

The repair depends on the diagnosis. Lack of coverage calls for new data or higher sampling weight on the relevant source. Negative transfer calls for source conditioning, adapter layers, or per-source filtering. Source conflict calls for schema repair and split redesign before another training run. This is why a mixture manifest should include not only weights, but also the intended role of each source: pretraining diversity, target-domain supervision, stress evaluation, or recovery examples.