Section 2.2: State, observation, hidden variables, partial observability

"My camera saw the block. My gripper discovered the friction. My log finally admitted both were incomplete."

A Probabilistic Robot Assistant

This section develops the vocabulary needed to debug almost every embodied AI system. A simulator may know the true pose, mass, friction, and contact state of every object. A robot policy may see only pixels, joint encoders, force readings, and a stale command queue. That difference is not a nuisance. It is the problem.

State is the information sufficient to predict future dynamics and reward when paired with an action. Observation is the sensor-facing evidence available to the agent. Hidden variables are state variables that matter but are not directly observed. Partial observability is the normal condition in which the latest observation is not enough.

Theory

Let $s_t$ be the world state and $o_t$ be the observation emitted by sensors or an environment wrapper. In a fully observable task, $o_t$ contains enough information to act as $s_t$. In embodied AI, that is usually false. Occlusion hides objects, contact reveals only local forces, latency makes images stale, and human intent is not directly measurable.

The agent therefore maintains an estimate $\hat{s}_t$ or a belief over possible states. This estimate may be a Kalman filter state, a particle filter, a learned recurrent hidden state, a transformer memory, or a structured world-model latent. The name matters less than the contract: what uncertainty does it represent, how is it updated, and how does action use it?

Worked Example

Code Fragment 2.2.1 updates a tiny belief state from a partial observation. The hidden variable is slip risk, which the camera cannot directly see.

# Section 2.2: runnable checkpoint for State vs. observation.
# Keep the output small so the evidence record can be inspected directly.
belief = {"block_x": 0.50, "block_visible": True, "slip_risk": 0.20}
observation = {"detected_x": 0.54, "visible": True, "force_spike": False}

if observation["visible"]:
    belief["block_x"] = 0.7 * belief["block_x"] + 0.3 * observation["detected_x"]
else:
    belief["block_visible"] = False

if observation["force_spike"]:
    belief["slip_risk"] += 0.25
else:
    belief["slip_risk"] *= 0.95

print({"estimated_x": round(belief["block_x"], 3), "slip_risk": round(belief["slip_risk"], 3)})

Expected output: an updated position estimate and slip-risk estimate. The example should make clear that vision updates visible pose, while contact evidence updates a hidden physical variable.

Practical Recipe

1. List variables needed to predict dynamics, not just variables found in the sensor packet.
2. Mark each variable as observed, estimated, delayed, hidden, or evaluator-only.
3. Add uncertainty to every estimated variable and log that uncertainty.
4. Carry history when the current observation is insufficient.
5. Evaluate under occlusion, sensor delay, calibration drift, and contact changes.

State-observation discipline prevents privileged information leakage. In simulation the evaluator may know object mass, contact normal, true pose, and collision margin. The policy should receive only the observation channels that a deployed system can provide. A result that mixes those views may measure access to simulator internals rather than intelligence.

The practical artifact is a variable ledger. Each variable is marked as observed, estimated, delayed, hidden, or evaluator-only. The ledger should also name the sensor or estimator that produces it and the uncertainty attached to it.

Build a leak test before training. The test should compare evaluator state fields with policy observation fields and fail when evaluator-only variables appear in policy input.

1. List all variables needed to predict dynamics and reward.
2. Mark each variable as observed, estimated, delayed, hidden, or evaluator-only.
3. Attach units, uncertainty, and source sensor or estimator.
4. Check that evaluator-only variables are absent from policy input.
5. Stress the belief with occlusion, delay, calibration drift, and contact changes.

# Build a variable ledger and detect privileged-state leakage.
variables = {
    "true_pose": "evaluator_only",
    "rgb_crop": "observed",
    "last_seen_pose": "estimated",
    "slip_risk": "hidden",
    "contact_force": "observed",
}
policy_input = {"rgb_crop", "last_seen_pose", "true_pose"}

def privileged_leaks(variables: dict[str, str], policy_input: set[str]) -> list[str]:
    return sorted(
        name for name in policy_input
        if variables.get(name) == "evaluator_only"
    )

print(privileged_leaks(variables, policy_input))

When behavior fails under partial observability, classify whether the agent lacked a sensor, lacked history, carried stale belief, underestimated uncertainty, or received leaked training information. Each cause points to a different repair.