This is a book for practitioners who build embodied systems and researchers who study them. It is written for the engineer shipping policies onto real robots and for the scientist pushing the frontier, at a density either will respect. It would rather show the equation, the algorithm, and the maintained tool than re-derive prerequisites you already have.
Who You Are
The book has a few concrete readers in mind.
- Robot-learning engineers training policies with RL or imitation, wiring up simulators, and fighting the sim-to-real gap on real hardware.
- Controls and robotics engineers moving into learning-based methods, who know kinematics, dynamics, and feedback control and want to add data-driven policies, world models, and VLAs to that foundation.
- Machine learning researchers entering embodied AI from vision, language, or generative modeling, who are fluent in deep learning but new to the closed loop, partial observability, and irreversible consequences.
- Autonomy teams in manipulation, locomotion, aerial, underwater, or driving, who need one coherent account of the modern stack instead of scattered papers and vendor docs.
The book assumes fluency with Python, comfort with tensors and array programming (NumPy and PyTorch or JAX), working probability (distributions, expectation, conditioning, basic estimation), and the basics of deep learning (gradients, backpropagation, training loops, common architectures). It also assumes you can read pseudocode and turn an algorithm into running code. Where a chapter leans on linear algebra, 3D geometry, probability and estimation, optimization, or PyTorch and JAX mechanics, it points to the appendices (A through D) for a focused refresher rather than re-teaching the material in the main text.
This is not a gentle introduction. If you are new to programming, or are meeting machine learning and deep learning for the first time, start with a foundational course or an earlier volume in the Hands-On AI Science series and return when the prerequisites above feel routine. The prior promise of self-contained undergraduate teaching has been retired: prerequisite material is assumed and refreshed only in the appendices, not built up from zero in the chapters. The payoff for that assumption is depth; the chapters spend their pages on the real mechanisms, the maintained tools, and the failure modes that actually occur, rather than on re-deriving what you already know.