"A robot that predicts badly either hesitates too long or commits too early. Both errors look like intelligence until the first contact event."
A Horizon-Aware Predictor
Predicting the Future is where a perception stack becomes a decision stack. These sections move from one-step dynamics to uncertainty, rollout horizon, and planning with predicted futures, always asking whether the forecast changes a real control or safety decision.
A useful predictive model earns its keep by improving a closed-loop metric such as stopping margin, contact timing, intervention count, or recovery rate. Open-loop fidelity is evidence, not the final objective.
Chapter Overview
Chapter 36 turns predictive modeling into an embodied-systems skill. The reader starts with why latency and partial observability force agents to reason ahead, then learns how state-space prediction differs from image reconstruction, why rollout error compounds with horizon, how uncertainty should gate trust, and how a planner consumes predicted futures.
The practical thread uses small inspectable probes, then points to maintained tools such as MuJoCo, Gymnasium, MJX, and logging stacks for real experiments. The theory thread stays tied to artifacts a lab would actually save: horizon-conditioned error tables, calibration plots, intervention logs, and matched closed-loop panels.
Prerequisites
Readers should be comfortable with state estimation, control loops, and the reinforcement-learning notation from Parts II through IV. Chapter 29 is especially helpful if belief updates and hidden-state estimation feel rusty.
Chapter Roadmap
- 36.1 Why agents need to predictMotivates prediction from latency, occlusion, and action delay, then ties the idea to closed-loop evidence.
- 36.2 Forward/dynamics models; state vs. observation predictionContrasts latent-state rollouts with pixel-space forecasting and shows when each target is useful for control.
- 36.3 Error accumulation and horizonExplains compounding rollout error, horizon selection, and why short trusted rollouts often beat long fantasy rollouts.
- 36.4 Uncertainty in predictionSeparates aleatoric from epistemic uncertainty and shows how calibration should influence planning.
- 36.5 Planning with predicted futuresConnects forecasts to receding-horizon action selection, risk-aware scoring, and robotics failure analysis.
Use hand-built probes to expose assumptions, then switch to maintained stacks when you need speed or scale. Good fits for this chapter include Gymnasium for environment contracts, MuJoCo or MJX for fast rollouts, PyTorch or JAX for model training, and experiment loggers such as Weights & Biases or plain versioned JSON artifacts for matched comparisons.
Hands-On Lab: Build A Horizon-Aware Prediction Panel
Objective
Build a small benchmark that trains one predictive model, evaluates horizon-conditioned error, measures calibration, and tests whether a short planning rule benefits from the learned forecast.
Skills
- Specify state, observation, horizon, and action contracts.
- Compare one-step and multi-step error on the same seed panel.
- Save a replayable artifact with traces, metrics, and failure labels.
Prerequisites
Python, NumPy, a simulator with deterministic resets, and a simple robot task with measurable latency or stopping constraints.
Steps
Step 1: Define targets
Decide whether the model predicts physical state, latent state, reward-relevant quantities, or raw observations. Write the choice down before training.
Step 2: Train a small predictor
Fit a one-step model and export the rollout script that can evaluate horizons 1 through H on held-out episodes.
Step 3: Measure uncertainty
Run an ensemble or dropout approximation and record mean error plus interval coverage.
Step 4: Attach a planner
Use the predictor to choose one-step or short-horizon actions, then compare the planner with a reactive baseline on the same panel.
Step 5: Write the postmortem
Label failures as data coverage, model bias, decoder mismatch, uncertainty miscalibration, or planner misuse.
Expected Result
A single folder with training config, held-out rollout traces, horizon error curves, calibration counts, and a short video set showing where prediction helped or hurt action selection.
Stretch Goals
Repeat the same panel with a latent-only predictor and a pixel decoder, then compare which target better supports control on the same matched metric.
This chapter works well as a course week because each section has a natural artifact: a one-step predictor, a horizon error table, an uncertainty calibration summary, and a small planner trace. The didactic move is to make students justify why a prediction target is action-relevant before they spend compute improving it.
The chapter is also where forecast length should become an explicit budget variable rather than a vague aspiration. Horizon length, model size, calibration method, and replanning rate all trade against one another, so the strongest reader takeaway is not "predict farther" but "predict only as far as the decision loop can trust."
Before leaving the chapter, the reader should be able to answer four questions: what is being predicted, why that target matters for action, how far ahead the model is trusted, and which artifact proves that trust is deserved.
Prediction belongs in an embodied course when it is tied to planning and safety. Forecasts without action consequences are perception exercises, not embodied decision-making.
For the application of these prediction models to model-based control and MBRL, see Chapter 37. Section 37.2 covers learned dynamics ensembles used for planning, and Section 37.3 covers CEM, MPPI, and latent MPC built on those models.
Bibliography & Further Reading
Foundational Papers, Tools, and References
Ha, D., and Schmidhuber, J.. "World Models." (2018). https://worldmodels.github.io/
A compact starting point for latent dynamics and control from imagined rollouts.
Hafner, D. et al.. "Learning Latent Dynamics for Planning from Pixels." (2019). https://arxiv.org/abs/1811.04551
PlaNet is the canonical reference for learning latent dynamics that are useful for planning rather than pure reconstruction.
Hafner, D. et al.. "Mastering Diverse Domains through World Models." (2023). https://arxiv.org/abs/2301.04104
DreamerV3 is the current baseline readers should know when connecting prediction to behavior improvement.
MuJoCo Documentation. "Overview." (accessed 2026). https://mujoco.readthedocs.io/
MuJoCo remains a practical simulator for prediction, state estimation, and model-based control experiments.
Farama Foundation. "Gymnasium Documentation." (accessed 2026). https://gymnasium.farama.org/
Gymnasium is a clean interface for experiments where reset, step, truncation, and seeding must stay explicit.