Section 53.2: Model uncertainty and calibration

A robust robot is not the one that never sees surprise, it is the one that notices surprise early enough to act differently.

A Runtime Monitoring Engineer

Why This Matters

Model uncertainty and calibration is useful only when it distinguishes disturbance sources and ties them to specific corrective actions. Robustness is not one scalar, it is a map from perturbation class to degraded behavior, detection delay, and residual risk.

A standard calibration statistic is the expected calibration error $$\mathrm{ECE} = \sum_{b=1}^{B} \frac{|S_b|}{n}\, |\mathrm{acc}(S_b) - \mathrm{conf}(S_b)|,$$ where $S_b$ is the set of predictions whose confidence falls in bin $b$. In embodied systems, calibration should be evaluated on action-relevant predictions, not only class labels.

Worked Example

A manipulation policy may be good at ranking candidate grasps yet still overclaim 0.95 confidence on cases that succeed only 0.70 of the time. A runtime gate built on that confidence will intervene too late.

predictions = [
    {"confidence": 0.9, "correct": 1},
    {"confidence": 0.8, "correct": 1},
    {"confidence": 0.8, "correct": 0},
    {"confidence": 0.6, "correct": 1},
    {"confidence": 0.55, "correct": 0},
    {"confidence": 0.95, "correct": 1},
]

def binned_ece(preds, n_bins=10):
    """Proper binned ECE: sum over bins of (n_b / N) * |accuracy_b - confidence_b|."""
    N = len(preds)
    edges = [i / n_bins for i in range(n_bins + 1)]
    ece = 0.0
    for i in range(n_bins):
        lo, hi = edges[i], edges[i + 1]
        in_bin = [p for p in preds if lo < p["confidence"] <= hi or (i == 0 and p["confidence"] == 0.0)]
        if not in_bin:
            continue
        n_b = len(in_bin)
        acc_b = sum(p["correct"] for p in in_bin) / n_b
        conf_b = sum(p["confidence"] for p in in_bin) / n_b
        ece += (n_b / N) * abs(acc_b - conf_b)
    return ece

ece = binned_ece(predictions)
print({"n_predictions": len(predictions), "binned_ece": round(ece, 4)})

{'n_predictions': 6, 'binned_ece': 0.15}

Expected output: The small gap here suggests the average scale is close, but a real evaluation would still inspect bins because cancellation can hide local miscalibration. Calibration is about the full reliability shape, not only the mean.

In embodied systems, calibration must be tied to consequences. A poorly calibrated collision predictor and a poorly calibrated object classifier are not equally serious if only one gates a safety-critical maneuver. In practice, teams often compare temperature scaling, isotonic regression, and conformal intervals on the exact signal that drives a stop, reroute, or human-review threshold.

The implementation contract is simple: the confidence value in the PyTorch or JAX tensor must be the same signal logged in the replay artifact, plotted in Weights & Biases or TensorBoard, and consumed by the ROS 2 gate. A calibration curve that is detached from the deployed threshold is analysis theater, not safety evidence.

A frequent failure is to calibrate on clean validation data and deploy on shifted scenes. The confidence scale then looks disciplined in the notebook and collapses in the field.

Cross-References

This section connects to Section 53.3 on OOD detection and Section 54.4 on shielded policies, where calibrated thresholds become actual intervention logic.