Section 4.1: Why space is the substrate of embodiment

"A robot without frames has many coordinates and no agreement."

A Meticulous Mapping Agent

Embodied AI is not image classification with motors attached. A tabletop robot must decide whether the mug is left of the gripper, whether the camera estimate is fresh enough for control, whether a planned base motion keeps the arm inside reach, and whether the simulator pose agrees with the controller pose. Those questions are spatial before they are learned.

The central habit is to write every spatial claim as a quantity plus its frame. A point in the camera frame, a velocity in the body frame, a pose in the world frame, and a normal vector in an object frame are not interchangeable arrays. They are contracts between perception, state estimation, planning, and control.

Theory

Write a point expressed in frame $A$ as $p^A$. Write a transform that maps coordinates from frame $B$ into frame $A$ as $T_{AB}$. Then the same physical target can be represented as $p^{camera}$ for perception, $p^{base}$ for reaching, and $p^{world}$ for navigation. The values differ because the basis differs; the target itself is the same:

$$p^A = T_{AB}\, p^B, \qquad p^B = T_{BA}\, p^A = (T_{AB})^{-1} p^A.$$

The transform is built from a rotation $R_{AB} \in SO(3)$ and a translation $t_{AB} \in \mathbb{R}^3$, the origin of frame $B$ expressed in frame $A$. Acting on a point this reads $p^A = R_{AB}\,p^B + t_{AB}$, so two frames give two coordinate triples for one physical target.

This notation is not decorative. It prevents a common production failure: an array crosses a subsystem boundary with no frame label, and the next subsystem silently assumes a different convention. The bug may survive unit tests because the numbers have the right shape.

Worked Example

Define two frames, a camera frame and a robot base frame, then represent one physical target in each. A camera detector reports the target in the camera frame; the gripper controller expects it in the base frame. The fragment prints both coordinate triples and confirms the round trip, exposing the contract that many full stacks hide under ROS 2 messages, simulator state, or library pose objects.

# Define two frames, represent one physical point in each, and print the transform.
# The round-trip back to the camera frame confirms the transform is consistent.
import numpy as np

# T_base_camera maps coordinates from the camera frame into the base frame.
T_base_camera = np.array([
    [0.0, -1.0, 0.0, 0.35],
    [1.0,  0.0, 0.0, 0.10],
    [0.0,  0.0, 1.0, 0.55],
    [0.0,  0.0, 0.0, 1.00],
])

p_camera = np.array([0.25, -0.10, 0.80, 1.0])   # target in the camera frame
p_base = T_base_camera @ p_camera               # same target in the base frame
p_camera_again = np.linalg.inv(T_base_camera) @ p_base  # round trip back

print("point in camera frame:", np.round(p_camera[:3], 3).tolist())
print("point in base frame:  ", np.round(p_base[:3], 3).tolist())
print("round-trip residual:  ", round(np.linalg.norm(p_camera - p_camera_again), 12))

Builder Recipe

1. Name the physical quantity before choosing a representation.
2. Attach a frame, timestamp, unit, and validity range to every spatial value.
3. Compute the minimal NumPy version and test one invariant by hand.
4. Replace the hand version with a maintained pose or transform library.
5. Log both the raw observation and the transformed action target in simulator or robot runs.

Production Pattern

Why space is the substrate of embodiment sits inside the Part II robotics contract: geometry defines where things are, kinematics defines what motion is possible, dynamics defines what motion costs, control defines how errors are corrected, and sensing defines what the agent can know on time.

Anchor every spatial claim in the action it enables: reach, avoid, grasp, localize, or explain a failure. This makes the section useful to students, builders, and researchers at the same time: the idea has an intuitive role, a formal interface, a runnable check, and a failure mode that can be reproduced.

Use this recipe when turning Why space is the substrate of embodiment into code, a simulator experiment, or a robot diagnostic. The point is not to use every library. The point is to keep the hand-built baseline and the maintained-tool path comparable.

1. Name every frame with a parent, child, unit convention, and timestamp policy.
2. Write one hand-checked transform chain and verify identity, inverse, and composition tests.
3. Run the same transform through ROS 2 tf2 or SciPy Rotation, then compare one point and one direction vector.
4. Record a frame audit with source sensor, latency, and expected sign convention.
5. Debug failed behavior by replaying the transform tree before changing policy or controller code.

Frame bugs start when a point, vector, pose, or timestamp is used without naming the coordinate system. Reproduce one world-to-body transform by hand before diagnosing perception or control.