Section 2.8: Why embodiment is usually partially observable

A Careful Control Loop

This section develops a physical explanation for why the agent-environment interface is rarely fully observable. In a simulator, full state may exist in memory. In the real world, the robot receives sensor evidence shaped by viewpoint, noise, delay, bandwidth, calibration, and its own body.

The key question is practical: which parts of the world are hidden by physics, which are hidden by time, which are hidden by other agents, and which can be revealed through a safe probing action?

Theory

Full observability would mean the current observation contains every variable needed for prediction and reward. Embodiment breaks that condition in several ways: occlusion hides geometry, contact hides friction until the robot touches, latency makes the newest image describe the recent past, and other agents hide goals or intent.

This does not make action impossible. It means the interface must support memory, uncertainty, and information-gathering actions. Sometimes the correct action is not "move toward the goal" but "move the camera," "touch gently," "wait one step," or "ask for clarification."

Worked Example

Code Fragment 2.8.1 turns partial observability into a debugging checklist. Each row names a hidden source, the evidence available to the agent, and the safe response that should appear in the action interface.

# Section 2.8: map hidden physical variables to probes and recovery actions.
# Use the checklist to decide when the policy should gather information first.
hidden_sources = [
    {"source": "occlusion", "evidence": "last_seen_pose", "probe": "move_camera"},
    {"source": "contact_friction", "evidence": "force_trend", "probe": "guarded_touch"},
    {"source": "latency", "evidence": "timestamp_age_ms", "probe": "predict_forward"},
    {"source": "human_intent", "evidence": "motion_cue", "probe": "wait_and_observe"},
]

for item in hidden_sources:
    action_mode = "probe_first" if item["source"] in {"occlusion", "human_intent"} else "act_with_monitor"
    print(f"{item['source']}: {action_mode} using {item['evidence']}")

Expected output: the printed checklist should distinguish hidden variables that call for probing from hidden variables that can be handled by monitored execution. If every row uses the same action mode, the interface is not yet using uncertainty.

Practical Recipe

1. List hidden variables by source: occlusion, contact, latency, calibration, internal hardware state, and other agents.
2. For each hidden variable, name the available evidence and the confidence field to log.
3. Add at least one information-gathering action for high-risk uncertainty.
4. Test policies under sensor delay, occlusion, friction change, and human-motion ambiguity.
5. Report recovery behavior separately from first-attempt success.

Partial observability becomes useful when it is tied to a closed-loop contract between policy, world, evaluator, and safety constraints. The contract names the hidden-variable inventory, observation stream, belief or confidence field, probing action, timing budget, safety boundary, and result artifact. That is the bridge between a readable concept and a system a skeptical builder can test.

For Why embodiment is usually partially observable, separate the conceptual claim, the systems claim, and the evidence claim. A good explanation, a clean API, and one successful rollout are different kinds of evidence, and the section should keep them distinct.

For Why embodiment is usually partially observable, a robust implementation starts with one inspectable baseline whose artifact records observations, actions, units, timestamps, seeds, termination reasons, and the perturbation applied. The maintained-tool version is useful only if it preserves that schema and lets the comparison remain construct-matched.

1. Write a one-paragraph task contract with observation, action, success, failure, and safety fields.
2. Start with the smallest simulator, dataset, or wrapper that exposes the task contract faithfully.
3. Run one deterministic smoke test and one perturbation test before scaling.
4. Save one artifact containing configuration, seed, metrics, traces, and failure labels.
5. Compare methods only when the same script evaluates the same panel, split, seed set, and metric.

When a partially observable embodied system fails, avoid labeling the whole method as weak. First assign the failure to occlusion, contact sensing, latency, calibration, internal hardware state, other-agent prediction, or evaluation. Then rerun one controlled perturbation that isolates the suspected cause. This pattern turns a disappointing rollout into a reusable diagnostic asset.