Part Overview
This part covers the geometry, kinematics, dynamics, control, and sensing that make physical agents intelligible. It connects formal ideas with the tools and labs needed to build working systems.
Chapters: 5. Each chapter includes theory, recipes, practical code, a library shortcut, and exercises.
Mathematical, Robotics, and Control Foundations gives the reader a working layer of the embodied AI stack. Later chapters assume this layer when agents must perceive, plan, act, and recover from mistakes.
This chapter develops spatial representation and coordinate frames as part of the embodied AI stack.
- 4.1 Why space is the substrate of embodiment
- 4.2 Points, vectors, poses, frames
- 4.3 Rotations: matrices, Euler angles, axis-angle, quaternions; pitfalls
- 4.4 Rigid transforms, homogeneous coordinates, SE(3)
- 4.5 2D and 3D transformations; transform trees (tf in ROS)
- 4.6 Camera, body, and world frames
- 4.7 Common frame mistakes and how to debug them
This chapter develops kinematics and robot motion as part of the embodied AI stack.
- 5.1 Position, velocity, acceleration; twists
- 5.2 Holonomic vs. non-holonomic motion
- 5.3 Differential-drive and car-like robots
- 5.4 Robot arms, joints, the kinematic chain
- 5.5 Forward kinematics
- 5.6 Inverse kinematics: analytic, numerical (Jacobian), and learned
- 5.7 Jacobians, singularities, manipulability
- 5.8 Motion constraints
This chapter develops dynamics and simulation math as part of the embodied AI stack.
- 6.1 From kinematics to dynamics: forces, torques, inertia
- 6.2 Rigid-body dynamics; the manipulator equation
- 6.3 Contact, friction, and why contact-rich sim is hard
- 6.4 Numerical integration and stability
- 6.5 Differentiable physics: what it buys you
- 6.6 Why GPU-parallel simulation changed robot learning
This chapter develops Control for AI Practitioners as part of the embodied AI stack.
- 7.1 Open-loop vs. closed-loop control
- 7.2 Feedback, error, stability, overshoot, oscillation
- 7.3 PID control, intuition and tuning
- 7.4 State-space control, LQR
- 7.5 Model predictive control (MPC) as receding-horizon optimization
- 7.6 Operational-space and whole-body control (preview for humanoids)
- 7.7 Controllers vs. policies; when learning helps and when it makes control unsafe
This chapter develops sensors, perception hardware, and state estimation as part of the embodied AI stack.
- 8.1 What sensors provide and what they cost
- 8.2 Cameras, depth (stereo/structured light/ToF), LiDAR
- 8.3 IMU, wheel odometry, joint encoders, proprioception
- 8.4 Tactile and force/torque sensing (GelSight, DIGIT): preview
- 8.5 Sensor noise and uncertainty models
- 8.6 Bayesian filtering: Kalman, EKF, particle filters
- 8.7 Sensor fusion intuition and practice
- 8.8 Perception as an imperfect window into the world
Part II Integration Map
Part II now carries one explicit robotics contract across every chapter: name the representation, state the timing and uncertainty assumptions, build the smallest inspectable baseline, use the maintained library path, and compare both paths with one shared evidence artifact.
| Chapter | Core Invariant | Practical Libraries | Evidence Check |
|---|---|---|---|
| Chapter 4: Spatial Representation and Coordinate Frames | coordinate frames, rigid transforms, SE(3), and transform-tree debugging | SciPy Rotation, ROS 2 tf2, spatialmath-python, Drake, OpenCV calibration | Most frame bugs look like weak perception or weak control. First test whether points, vectors, poses, and timestamps were transformed through the same frame tree. |
| Chapter 5: Kinematics and Robot Motion | twists, constraints, forward kinematics, inverse kinematics, Jacobians, and manipulability | Pinocchio, Robotics Toolbox for Python, MoveIt 2, Drake, ROS 2 control | Kinematic failures often arrive as a plausible pose with an impossible motion. Inspect joint limits, Jacobian rank, constraint assumptions, and frame choice before blaming the planner. |
| Chapter 6: Dynamics and Simulation Math | forces, torques, inertia, contact, integration, differentiable physics, and GPU-parallel simulation | MuJoCo, MJX, Drake, Pinocchio, Isaac Lab | A simulation that looks smooth can still be physically misleading. Check timestep sensitivity, contact stiffness, damping, friction cones, and actuator saturation before trusting policy performance. |
| Chapter 7: Control for AI Practitioners | feedback, stability, PID, LQR, MPC, operational-space control, policies, and safety filters | python-control, CasADi, Drake, do-mpc, ROS 2 control | A learned policy can hide an unstable inner loop until the disturbance changes. Check update rate, delay, saturation, integral windup, and fallback behavior before scaling training. |
| Chapter 8: Sensors, Perception Hardware, and State Estimation | sensor models, calibration, noise, Bayesian filtering, fusion, latency, and partial observability | OpenCV, ROS 2 robot_localization, FilterPy, Kalibr, Open3D | Sensor failures often masquerade as bad planning. Check calibration drift, unsynchronized clocks, frame mismatch, covariance overconfidence, and latency before changing the policy. |
Geometry says where, kinematics says what motion is possible, dynamics says what motion costs, control says how errors are corrected, and estimation says what the agent can know on time.
Choose one embodied task and fill one row for every Part II layer: frame, motion model, dynamics assumption, controller, sensor model, latency budget, library shortcut, and failure label. The row is complete only when all layers share the same task, seed, split, and metric artifact.
What's Next?
After this part, Part III: Simulation, Tooling, and the Modern Stack extends the stack.