Section 10.1: Gym is dead; Gymnasium is the standard

A Careful Control Loop

What This Section Builds

The migration from legacy Gym to Gymnasium is operational, not cosmetic. The important change is the environment contract that every later RL script, simulator wrapper, benchmark, and debugging trace will assume.

The goal is a reproducible habit: call reset(seed=...), unpack step into five values, treat terminated and truncated as different signals, and save enough info to explain what happened.

Theory

Gymnasium keeps the familiar environment idea from Gym but modernizes the contract. A single-agent environment resets to (observation, info). Each step returns (observation, reward, terminated, truncated, info). The extra flag matters because a task can end because the robot achieved or failed the objective, or because an external limit stopped the episode before the task itself reached a terminal state.

For learning code, that distinction controls bootstrapping and evaluation. A policy update may treat a true terminal state differently from a time limit. For embodied systems, it also controls incident analysis: falling over, reaching the goal, running out of time, and hitting a safety boundary should not collapse into one vague done bit.

Worked Example

Code Fragment 10.1.1 below uses the current Gymnasium API on a small control task. The same unpacking pattern carries over to robot simulators, where info should hold diagnostic fields such as contact state, time limit source, or safety margin.

# Inspect the modern Gymnasium reset and step contract.
# The five step fields separate task endings from time-limit endings.
import gymnasium as gym

env = gym.make("CartPole-v1")
observation, info = env.reset(seed=7)
env.action_space.seed(7)

action = env.action_space.sample()
next_observation, reward, terminated, truncated, info = env.step(action)

print(type(observation).__name__, observation.shape, info)
print(action, float(reward), terminated, truncated)
env.close()

The expected output is a four-value observation vector, an empty initial info dictionary, and one sampled step with reward 1.0 while both ending flags remain false. Read that combination as evidence that the Gymnasium loop is returning the modern five-field contract and that this particular first step did not end the episode for either task or time-limit reasons.

Practical Recipe

1. Use import gymnasium as gym in new code and update legacy examples during migration.
2. Call env.reset(seed=seed) before the first step and unpack both the observation and info.
3. Unpack env.step(action) into five fields every time.
4. Reset when terminated or truncated is true, but log which flag caused the reset.
5. Store env.spec.id, wrapper stack, seed, render mode, and library versions with the result artifact.

Gymnasium is the standard because current RL libraries, environment suites, and Farama documentation converge on its contract. The migration is not cosmetic: it changes how an experiment represents episode endings, reset information, render modes, environment metadata, and wrapper behavior.

The builder's discipline is to treat an environment loop as a typed interface. If an embodied policy is evaluated through the wrong unpacking pattern, the algorithm may still train, but the result artifact cannot answer the basic scientific question: what caused each episode to stop?

A robust migration starts with one old loop and one new loop evaluated on the same environment seed. The comparison is valid only if both loops record the same episode boundary fields and the Gymnasium version preserves the distinction between terminated and truncated.

1. Find every call site that unpacks done.
2. Rewrite the loop to unpack terminated and truncated.
3. Add a one-episode smoke test that asserts the output tuple has five fields.
4. Save counts for termination and truncation separately.
5. Only then reconnect the loop to a trainer or evaluation dashboard.

# Verify that a migrated Gymnasium loop is deterministic under a seed.
# Same seed should reproduce the first sampled action and first transition.
import gymnasium as gym

def first_step(seed):
    env = gym.make("CartPole-v1")
    observation, info = env.reset(seed=seed)
    env.action_space.seed(seed)
    action = env.action_space.sample()
    next_observation, reward, terminated, truncated, info = env.step(action)
    env.close()
    return round(float(next_observation[0]), 5), int(action), terminated, truncated

print(first_step(21))
print(first_step(21))
print(first_step(22))

The expected output repeats the first tuple exactly for the repeated seed and changes both the sampled action and next observation when the seed changes. That is the minimal sign that environment reset seeding and action-space seeding are both wired correctly.

When an experiment about gym is dead; gymnasium is the standard fails, avoid labeling the whole method as weak. First assign the failure to perception, state estimation, planning, control, timing, data coverage, or evaluation. Then rerun one controlled perturbation that isolates the suspected cause. This pattern turns a disappointing rollout into a reusable diagnostic asset.