Section 10.5: Rendering, logging, and debugging

A Careful Control Loop

What This Section Builds

Rendering, logging, and debugging are the evidence path for an environment. Rendering shows what the environment believes is happening, logging preserves what happened, and debugging connects those records to a concrete failure cause.

The goal is to stop treating a reward curve as the whole story. An embodied environment should produce enough trace evidence to answer which observation arrived, which action was sent, what the environment returned, and why the episode ended.

Theory

Gymnasium environments declare render modes such as human, rgb_array, or ansi. The right mode depends on the artifact: a live window is useful for local debugging, an RGB array can be saved as video, and text rendering can be checked in automated tests.

Logging should sit next to the environment loop rather than after training. Each step record should include step index, seed, action, reward, termination flag, truncation flag, and selected info fields. For robotics, add controller status, contact events, safety margins, and timing.

Worked Example

Code Fragment 10.5.1 uses an ansi render mode so the example works without a graphics window. The render frame gives a human-readable view, while the step return gives the machine-readable trace.

# Use text rendering when a debug check should run without a GUI.
# The step trace still records reward, ending flags, and info keys.
import gymnasium as gym

env = gym.make("FrozenLake-v1", render_mode="ansi", is_slippery=False)
observation, info = env.reset(seed=3)
frame = env.render()
visible = [line for line in frame.splitlines() if line.strip()]

observation, reward, terminated, truncated, info = env.step(1)
clean_row = visible[0].replace("\x1b[41m", "[").replace("\x1b[0m", "]")

print(clean_row)
print({"obs": int(observation), "reward": reward, "ended": terminated or truncated, "info_keys": sorted(info.keys())})
env.close()

The expected output combines a human-readable render frame with a machine-readable step record. The frame shows the current grid state, while the dictionary confirms that the sampled transition did not end the episode and that a probability diagnostic is available in info.

Practical Recipe

1. Choose a render mode that matches the artifact: live inspection, saved video, image array, or text trace.
2. Log one row per environment step with action, reward, ending flags, and selected info.
3. Save the wrapper stack and render mode with the log.
4. When a rollout fails, classify the failure before changing the policy.
5. Keep two representative failure traces for each reported metric table.

Rendering and logging answer different parts of the same question. Rendering says what the environment displays as happening. Logging says what the algorithm saw and optimized. A strong debugging workflow keeps those synchronized by seed and step index.

The graduate-level habit is to require traceability from a reported number back to at least one representative episode. A success rate without failure traces is fragile because it cannot show which assumptions survived contact with the simulator.

A robust debugging implementation starts with a tiny trace format. The trace should be small enough to inspect by hand and structured enough to join with videos, metrics, and safety events.

1. Choose the minimal render mode that captures the failure evidence.
2. Write one log row per step before training long runs.
3. Include terminated, truncated, and selected info fields in each row.
4. Save seeds and wrapper stack beside the trace.
5. Review a few failure traces before tuning reward or model architecture.

# Record a compact step trace that can be inspected after rollout.
# Each row preserves reward, ending status, and diagnostic keys.
import gymnasium as gym

env = gym.make("CartPole-v1")
observation, info = env.reset(seed=17)
env.action_space.seed(17)
trace = []

for step_index in range(3):
    action = env.action_space.sample()
    observation, reward, terminated, truncated, info = env.step(action)
    trace.append({
        "step": step_index + 1,
        "action": int(action),
        "reward": float(reward),
        "ended": terminated or truncated,
        "info_keys": sorted(info.keys()),
    })

print(trace)
env.close()

The expected output is a short rollout ledger with one dictionary per step. Read it as a minimal debugging artifact: every action, reward, and ending flag is preserved in order, so a later aggregate return can still be traced back to concrete behavior.

When an experiment about rendering, logging, and debugging fails, avoid labeling the whole method as weak. First assign the failure to perception, state estimation, planning, control, timing, data coverage, or evaluation. Then rerun one controlled perturbation that isolates the suspected cause. This pattern turns a disappointing rollout into a reusable diagnostic asset.