Section 2.3: Action types: discrete, continuous, symbolic, motor-level, chunked

"Choosing an action space is how you tell a robot what kinds of mistakes it is allowed to make."

A Cautious Policy Interface

This section develops a design checklist for action representation. Action spaces are not interchangeable wrappers around a model. They define controllability, safety, latency, data requirements, transfer difficulty, and how quickly an agent can recover from an error.

Discrete actions are easy to enumerate, symbolic actions are useful for planning, continuous actions match motors and physics, motor-level actions expose control detail, and chunked actions reduce inference frequency while increasing commitment. OpenVLA-style systems add another pattern: map image and language context to action tokens or continuous control heads that must still respect robot limits.

Theory

Let the action space be $\mathcal{A}$. A discrete $\mathcal{A}$ might contain actions such as open, close, or move-left. A continuous $\mathcal{A}$ might be a vector of joint torques, velocities, or end-effector pose deltas. A symbolic $\mathcal{A}$ might contain calls such as pick(red_block). A chunked $\mathcal{A}$ contains sequences of low-level actions emitted at once.

The right action space depends on embodiment and timing. A high-level action can be easier to learn but hides safety-critical details. A low-level action can be precise but makes long-horizon reasoning harder. A chunked action can smooth robot motion and reduce model calls, but it delays correction if the world changes mid-chunk.

Worked Example

Code Fragment 2.3.1 compares four action representations for the same tabletop instruction. Notice that each representation shifts responsibility to a different layer of the system.

# Section 2.3: runnable checkpoint for Action types: discrete, continuous, symbolic, motor-level,
# chunked.
# Keep the output small so the evidence record can be inspected directly.
action_spaces = {
    "discrete_skill": ["find_object", "grasp", "place"],
    "symbolic_call": "place(red_block, tray)",
    "continuous_delta": {"dx_m": 0.01, "dy_m": -0.02, "dz_m": 0.00, "grip": 0.7},
    "chunked_delta": [
        {"dx_m": 0.01, "grip": 0.5},
        {"dx_m": 0.01, "grip": 0.7},
        {"dx_m": 0.00, "grip": 0.9},
    ],
}
for name, action in action_spaces.items():
    print(name, action)

Practical Recipe

1. Start from the actuator, safety monitor, and controller rate, then move upward to skills.
2. Choose the coarsest action that still allows timely recovery.
3. Record units, bounds, coordinate frame, rate, and clipping behavior.
4. Test action latency by injecting delay and measuring recovery.
5. Report action validity failures separately from task failures.

Action representation is an architectural boundary. A symbolic skill shifts burden to a planner and skill library. An end-effector delta shifts burden to a controller and calibration stack. A joint command shifts burden to the learned policy and safety monitor. A chunked action shifts burden to prediction because the policy commits before seeing every intermediate consequence.

The practical question is not which action type is most elegant. It is which layer should own timing, contact, validity checking, and recovery for the task at hand.

Audit an action interface before training. The audit should fail if units, frames, rates, bounds, or chunk semantics are absent.

1. Name the action level: symbolic, skill, end-effector, joint, torque, velocity, or chunked sequence.
2. Record units, coordinate frame, bounds, update rate, controller below the action, and clipping behavior.
3. Define what happens when a command is invalid or stale.
4. Inject saturation, delay, and mid-chunk observation changes.
5. Report action validity failures separately from policy-task failures.

# Audit an action interface for fields needed by a real controller.
action_interface = {
    "level": "end_effector_delta",
    "units": {"dx": "m", "dy": "m", "dz": "m", "yaw": "rad"},
    "frame": "tool0",
    "rate_hz": 20,
    "bounds": {"dx": 0.02, "dy": 0.02, "dz": 0.015, "yaw": 0.10},
    "clip_behavior": "clip_and_log",
    "controller_below": "cartesian_impedance",
}

def missing_action_contract(interface: dict[str, object]) -> list[str]:
    required = ["level", "units", "frame", "rate_hz", "bounds", "clip_behavior", "controller_below"]
    return [key for key in required if key not in interface]

print(missing_action_contract(action_interface))

When an action interface fails, ask whether the command was invalid, clipped, stale, in the wrong frame, too coarse, too low-level, or too committed through chunking. Those are different failures and should not be collapsed into "bad policy."

pip install numpy pandas

required = {"level", "units", "frame", "rate_hz", "bounds", "clip_behavior"}
missing = sorted(required - contract.keys())
assert not missing, f"Action contract missing fields: {missing}"
print({"contract_ready": True, "rate_hz": contract["rate_hz"], "bounds": contract["bounds"]})

command = {"dx": 0.05, "dy": 0.00, "dz": 0.00, "grip": 0.6}
clipped = {**command, "dx": min(command["dx"], contract["bounds"]["dx"])}
clip_amount = command["dx"] - clipped["dx"]
assert clip_amount >= 0.0
print({"raw_dx": command["dx"], "executed_dx": clipped["dx"], "clip_amount": round(clip_amount, 3)})

interfaces = [
    {"name": "single_delta", "chunk_len": 1, "rate_hz": 20},
    {"name": "five_step_chunk", "chunk_len": 5, "rate_hz": 20},
]
for item in interfaces:
    item["commitment_ms"] = 1000 * item["chunk_len"] / item["rate_hz"]
print(interfaces)