Section 17.3: Isaac Lab with SKRL / rl_games / RSL-RL

A Careful Control Loop

For Isaac Lab with SKRL / rl_games / RSL-RL, GPU RL depends on simulator fidelity, PPO rollout semantics, reward terms, and reset distribution being versioned in the same training artifact.

This section develops the contract between an Isaac Lab environment and a training runner. The contract is concrete: observation groups, action clipping, reward terms, reset buffers, device, runner config, and evaluation script.

The key question is practical: if you switch from RSL-RL to rl_games or SKRL, which parts of the experiment are allowed to change, and which parts must remain identical for the comparison to mean anything?

Theory

Isaac Lab tasks commonly expose named observation groups. A policy actor may receive proprioception, commands, and history, while the critic may receive privileged simulator state such as terrain height, contact flags, or object poses. In asymmetric actor-critic training, those groups must be routed explicitly so privileged information helps value learning without leaking into the deployed actor.

The runner boundary also controls performance. rl_games can work directly with GPU buffers, RSL-RL expects its own rollout storage conventions, and SKRL emphasizes transparent algorithm configuration across backends. The task is the scientific object; the runner is the optimizer and storage implementation.

Worked Example

Code Fragment 17.3.1 makes the runner choice explicit. It is a small manifest, but it captures the fields that are often hidden in launcher commands and YAML files.

# Compare Isaac Lab runner wrappers by the contract they must preserve.
# The task stays fixed while storage, observation packing, and logging vary.
from dataclasses import dataclass

@dataclass
class RunnerContract:
    runner: str
    wrapper_role: str
    actor_input: str
    critic_input: str

    def as_row(self) -> dict[str, object]:
        return asdict(self)

contracts = [
    RunnerContract("RSL-RL", "rollout storage for locomotion PPO", "obs", "privileged_obs"),
    RunnerContract("rl_games", "GPU buffer bridge and clipping", "obs", "states"),
    RunnerContract("SKRL", "readable algorithm and memory config", "states", "state_values"),
]

for contract in contracts:
    print(f"{contract.runner}: actor={contract.actor_input}, critic={contract.critic_input}")

Expected output: the trace should reveal which tensor group reaches the actor and which reaches the critic. If the manifest cannot answer that question, an asymmetric training result is not reproducible.

Practical Recipe

1. Define the Isaac Lab task first: robot asset, scene, observations, actions, rewards, terminations, curriculum, and randomization.
2. Choose the runner based on the experiment goal: speed, readable algorithm research, recurrent policies, or compatibility with existing locomotion configs.
3. Record actor observation groups and critic-only privileged groups before training.
4. Keep train and play scripts separate so evaluation uses deterministic actions and held-out seeds.
5. Export the checkpoint, normalization statistics, runner config, task config, and exact Isaac Lab commit or release together.

The idea in this section becomes useful when the runner boundary is explicit. Isaac Lab gives you a task graph; the runner gives you a learner. The wrapper is where the two meet, so it must be part of the experiment record.

The graduate-level habit is to separate task validity from runner performance. A runner can update faster without improving the task definition, and a better task curriculum can improve every runner. A fair comparison changes one of those layers at a time.

A robust implementation starts with a manifest that binds task, wrapper, runner, and evaluation. The manifest is small enough to review in a pull request and concrete enough to reproduce a run later.

1. Record the task entry point and runner wrapper in the same config artifact.
2. List actor observations and critic-only observations separately.
3. Store action clipping, observation clipping, and normalization settings.
4. Save train and play commands with explicit seed panels.
5. Compare runners only through the same play script and metric exporter.

# Record the Isaac Lab task, wrapper, and evaluation contract together.
# This prevents runner comparisons from hiding observation or seed changes.
from dataclasses import dataclass, asdict

@dataclass
class IsaacLabRunManifest:
    task: str
    runner: str
    wrapper: str
    actor_obs: tuple[str, ...]
    critic_obs: tuple[str, ...]
    eval_panel: str

    def as_row(self) -> dict[str, object]:
        return asdict(self)

manifest = IsaacLabRunManifest(
    task="Isaac-Velocity-Rough-Anymal-D-v0",
    runner="rsl_rl",
    wrapper="RslRlVecEnvWrapper",
    actor_obs=("proprioception", "commands", "history"),
    critic_obs=("terrain_heights", "contact_flags", "base_velocity"),
    eval_panel="rough_terrain_holdout_256",
)
print(manifest.as_row())

When an Isaac Lab run fails, inspect the wrapper contract before changing rewards. Common faults include critic-only state leaking into actor inputs, train-time randomization missing from evaluation, action clipping differing across runners, and normalization files not loaded during play.