Section 55.2: Real-time inference and control rates

For Real-time inference and control rates, deployment quality is measured by the command stream, safety monitor state, and replayable evidence behind each command.

A Careful Control Loop

Problem First

A strong model can be unusable if inference arrives after the controller needed the decision.

The practical question is therefore specific: which observation arrives, which state estimate is trusted, which action is allowed, which monitor can interrupt it, and which artifact proves the claim afterward?

Theory

Real-time deployment is a multirate systems problem. A stabilizing controller may run at 200 to 1000 Hz, a state estimator at 30 to 200 Hz, a learned visuomotor policy at 5 to 30 Hz, and a task planner at 0.5 to 2 Hz. These loops must exchange commands without violating freshness constraints.

A useful contract is

$$\Pr(\tau_{\mathrm{age}} > \tau_{\max}) \le \epsilon,\qquad \tau_{\mathrm{age}}=\tau_{\mathrm{sense}}+\tau_{\mathrm{queue}}+\tau_{\mathrm{infer}}+\tau_{\mathrm{publish}}.$$

Average latency is not enough. Physical instability is typically triggered by the tail of the latency distribution, so p95 and p99 command age belong in the same artifact as task success.

Worked Example

A legged robot often stabilizes at high rate while a vision module and a policy run much more slowly. The safe pattern is to hold the low-level loop constant and explicitly decide when a slower policy output is still fresh enough to consume.

from statistics import quantiles

control_period_ms = 5
command_ages_ms = [28, 32, 35, 31, 40, 29, 36, 34, 52, 33]
max_freshness_ms = 40

p95 = quantiles(command_ages_ms, n=20)[18]
deadline_miss_rate = sum(age > max_freshness_ms for age in command_ages_ms) / len(command_ages_ms)
degraded_mode = p95 > max_freshness_ms

report = {
    "section": "55.2",
    "control_period_ms": control_period_ms,
    "policy_age_p95_ms": round(p95, 1),
    "deadline_miss_rate": deadline_miss_rate,
    "degraded_mode": degraded_mode,
}
print(report)

{'section': '55.2', 'control_period_ms': 5, 'policy_age_p95_ms': 57.4, 'deadline_miss_rate': 0.1, 'degraded_mode': True}

The expected output should make the control decision obvious. Here the p95 command age exceeds the freshness threshold, so the correct interpretation is not merely that latency is "a bit high" but that the stack should enter a degraded mode or reduce reliance on the slow policy.

Practical Recipe

1. Write the observation, action, monitor, metric, and artifact fields before selecting a model.
2. Run a deterministic smoke test and one named perturbation from the panel.
3. Log success, safety events, latency, energy or resource use, and recovery status in the same row group.
4. Compare only methods evaluated by the same script on the same panel and seed plan.
5. Attach a short postmortem to each failed rollout so the artifact remains useful after the plot is forgotten.

Real-time inference and control rates becomes operational when the metric is tied to a runtime interface. The interface names the sensor stream, state estimate, action representation, timing budget, safety or robustness monitor, and deployment artifact.

The disciplined habit is to separate three claims. The conceptual claim explains why the method should help. The systems claim explains which interface it changes. The evidence claim records which measurement would convince a skeptical builder.

Cross-References

For Real-time inference and control rates, connect benchmark design, sim-to-real transfer, uncertainty, and safety barriers through the deployment artifact that will be checked before release.

When a rate contract fails, classify the fault as sensor backlog, estimator lag, inference stall, middleware queue growth, scheduler preemption, or controller integration error. Then isolate that mechanism with one targeted perturbation such as synthetic GPU contention or callback bursts.