For Application Track Capstone Templates, field deployment evidence must include motors, logs, limits, monitors, and the consequence of each intervention.
A Systems-Minded Embodied AI Agent
A capstone becomes a research-grade embodied AI project when it has an application track, a starter stack, an evidence panel, a safety constraint, and a reproducibility card.
Why This Section Was Added
This application layer closes the gap between textbook breadth and the daily needs of researchers and builders. In research and builder capstones, the core question is not whether one component scores well in isolation. The question is whether the system produces an action, a safety boundary, and an evidence artifact that another team can inspect.
The central contract is compact: define the operating domain, name the state variables, state the action interface, identify the safety monitor, and save the log that proves what happened. Every serious embodied system eventually becomes this contract, whether it is a drone, an autonomous vehicle, a humanoid, a mobile manipulator, an industrial fleet, or a simulator-first research platform.
Choose the model after the evidence contract is clear. A stronger model cannot rescue missing calibration, unclear frames, unbounded actions, stale maps, or metrics computed on incompatible scenario panels.
Technical Core
For this section, the working mathematical object is:
$$P=(O,A,S,M,E,R).$$
The notation is intentionally a system contract rather than a single loss function. It ties the learned or planned output to state, action, environment constraints, and the measured evidence. A leading researcher can replace the simple expression with a detailed estimator, controller, simulator, or assurance argument without changing the structure of the artifact.
- Define the operating domain, robot interface, state variables, and safety constraints.
- Choose one scenario panel and keep it fixed while comparing baselines and shortcuts.
- Run the hand-built baseline and the maintained tool path on the same configuration.
- Save logs, metrics, latency, failure labels, and replay artifacts in one manifest.
- Promote the method only if the action, safety boundary, or recovery behavior improves.
Practical Stack
The practical tool stack for this section is: Isaac Lab, MuJoCo, CARLA, CommonRoad, PX4 SITL, Habitat 3.0, ManiSkill, ROS-Industrial, Open-RMF, LeRobot. The teaching path should start with a small inspectable baseline, then shift to maintained libraries once the mechanism is clear. The shortcut is valuable because it handles optimized kernels, standard data formats, timing integration, visualization, and deployment hooks that hand code usually handles poorly.
| Layer | What To Specify | Evidence To Save |
|---|---|---|
| Operating domain | Environment, weather or scene limits, human zones, task envelope, and excluded cases. | ODD card or site card. |
| State and actions | Frames, units, rates, uncertainty, command limits, and fallback behavior. | Interface manifest and sample logs. |
| Evaluation | Scenario panel, metric code, seeds, perturbations, and failure taxonomy. | One construct-matched result artifact. |
| Deployment | Monitoring, incident response, rollback, calibration checks, and maintenance cadence. | Safety case, incident report, and replay case. |
Stress the system with oversized scope, missing baseline, simulator-only claims, unpaired metrics, vague safety constraints, and missing artifact cards. These are not edge-case decorations. They are the normal conditions that separate a publishable demo from a deployable embodied system.
Consider a reader chooses one of ten application tracks, builds a small system, and submits the same evidence package format used by the rest of the book. A useful implementation logs the observation stream, state estimate, chosen action, safety monitor status, controller status, and post-event recovery. That log keeps the team from blaming the model when the true fault is calibration, timing, planning, control, or evaluation.
# Build one application evidence card for Section 59.12.
from dataclasses import dataclass, asdict
@dataclass
class ApplicationEvidence:
section: str
operating_domain: str
state_action_contract: str
tool_stack: str
perturbation: str
metric: str
replay_artifact: str
def as_row(self) -> dict[str, object]:
return asdict(self)
card = ApplicationEvidence(
section="59.12",
operating_domain="research and builder capstones",
state_action_contract="frames, units, rates, limits, safety monitor",
tool_stack="Isaac Lab, MuJoCo, CARLA, CommonRoad, PX4 SITL, Habitat 3.0, ManiSkill, ROS-Industrial, Open-RMF, LeRobot",
perturbation="oversized scope",
metric="same-panel task success plus safety and recovery labels",
replay_artifact="config, log, metric output, and failure case",
)
print(card.as_row())
{'section': '59.12', 'operating_domain': 'research and builder capstones', 'state_action_contract': 'frames, units, rates, limits, safety monitor', 'tool_stack': 'Isaac Lab, MuJoCo, CARLA, CommonRoad, PX4 SITL, Habitat 3.0, ManiSkill, ROS-Industrial, Open-RMF, LeRobot', 'perturbation': 'oversized scope', 'metric': 'same-panel task success plus safety and recovery labels', 'replay_artifact': 'config, log, metric output, and failure case'}The expected output is a capstone template card that already exposes the main grading risks before any model training starts. If the printed record shows an oversized scope but no replay artifact or no construct-matched metric, the project is still a topic idea rather than a capstone that another reader can run and evaluate.
The hand-built evidence card is only a few lines, but production work should let Isaac Lab, MuJoCo, CARLA, CommonRoad, PX4 SITL, Habitat 3.0, ManiSkill, ROS-Industrial, Open-RMF, LeRobot handle standard interfaces, logs, simulators, controllers, and visualizers. The reduction is from dozens of fragile glue-code lines to a maintained stack plus one manifest, while preserving the evidence schema.
Recipe For Builders
- Write the operating-domain card before training, tuning, or route planning.
- Choose a baseline that is simple enough to debug by eye.
- Add the maintained tool path and keep the output schema identical.
- Run one nominal case, one degraded-sensing case, one recovery case, and one safety-boundary case.
- Ship the result only with logs, configuration, metric code, and a replayable failure case.
For application track capstone templates, the useful test is simple: could a teammate point to the log line, plot, or trace that proves the idea changed the agent's next action?
Can you state the operating domain, state variables, action interface, safety monitor, perturbation, and replay artifact for research and builder capstones without opening another file? If not, the system is not yet specified.
The frontier is moving toward open robot foundation models, large-scale simulators, richer datasets, formal safety cases, and fleet telemetry. The durable research contribution is the evidence loop that connects those pieces without hiding assumptions.
Application Track Capstone Templates belongs in the book because it turns an application domain into a reproducible embodied AI build path: theory, tool stack, scenario panel, safety constraint, and replayable evidence.
select one application track and fill objective, assumptions, stack, metrics, safety constraints, evidence artifacts, and grading rubric before writing code. Submit the result as one evidence card, one metric artifact, and one failure replay note.
Topic-Native Deepening
This section is the synthesis layer for the whole capstone chapter. Instead of one project, it offers reusable track templates that cover the main embodied-application families and make their hidden systems assumptions visible before a team starts coding.
The design problem is curricular and technical at the same time: how do you give students enough structure to build something real without flattening the differences between drones, autonomous driving, humanoids, household robots, and industrial fleets?
Application Track Capstone Templates becomes teachable once the student can state the operative variables, the decision boundary, and the evidence artifact. The section should therefore be read together with Chapter 47 on drones, Chapter 48 on autonomous driving, and Chapter 46 on humanoids, where the same loop is developed from adjacent angles.
For each application track define a template $T=(\text{ODD},\text{state},\text{actions},\text{safety},\text{metrics},\text{artifacts})$. The template succeeds when teams can instantiate it to different domains without changing the evidence logic.
The power of the template is invariance. The operating domain and controller differ across tracks, but every good embodied capstone still needs a task card, a safety envelope, a metric panel, and replayable evidence.
- Select a domain, such as household manipulation, drone inspection, autonomous driving, or warehouse autonomy.
- Specify the operating-domain card, state variables, action interface, and safety constraints.
- Choose a baseline stack and one proposed improvement path.
- Commit to one metric panel and one failure taxonomy before implementation.
- Deliver the track-specific artifact bundle plus a common evidence card.
| Dimension | What To Specify | Why It Matters |
|---|---|---|
| Household manipulation | LeRobot, ManiSkill, ROS 2, tabletop or home scene | Success, retries, object damage, recovery quality. |
| Drone inspection | PX4 SITL, ROS 2, perception stack, route planner | Coverage, battery reserve, geofence events, missed defects. |
| Autonomous driving | CARLA, CommonRoad, planner, controller | Route completion, comfort, infractions, fallback quality. |
| Humanoid or mobile manipulation | Isaac Lab, MuJoCo, whole-body control stack | Task success, balance loss, recovery, safety stops. |
def validate_track(payload: dict[str, object]) -> dict[str, object]:
assert payload, "payload must not be empty"
return payload
# Track-instantiation summary.
track = {
"domain": "autonomous driving research prototype",
"odd": "campus roads, daylight only, 20 km/h max",
"safety_constraint": "geofenced route plus emergency stop",
"artifact_bundle": ["scenario panel", "metrics", "replay", "incident note"],
}
print(validate_track(track))
{'domain': 'autonomous driving research prototype', 'odd': 'campus roads, daylight only, 20 km/h max', 'safety_constraint': 'geofenced route plus emergency stop', 'artifact_bundle': ['scenario panel', 'metrics', 'replay', 'incident note']}The expected output should feel like a ready-to-use project brief. If the operating domain and artifact bundle are explicit, a team can start building without arguing over what counts as evidence.
After the from-scratch contract is clear, the practical route uses Isaac Lab, MuJoCo, CARLA, CommonRoad, PX4 SITL, Habitat 3.0, ManiSkill, ROS-Industrial, Open-RMF, LeRobot. The payoff is that standard interfaces, logging, batching, and replay support move from ad hoc glue code into maintained infrastructure, while the evidence schema stays the same.
Instructors can assign different tracks to different teams while keeping one shared grading rubric because the evidence schema remains common. That is what makes the chapter scalable as a course resource.
A useful research extension is cross-track transfer, for example whether a memory or planning module developed in household robotics survives adaptation to drones or warehouse fleets.
For capstone templates, the artifact should show that the chosen application track has a measurable success predicate, a failure taxonomy, and a reproducible evaluation route.
Section References
Isaac Lab. https://isaac-sim.github.io/IsaacLab/
Simulation and robot-learning platform for capstone projects.
MuJoCo Playground. https://playground.mujoco.org/
Reference for fast embodied control experimentation.
CARLA. https://carla.org/
Autonomous driving simulator for closed-loop driving capstones.
PX4. https://docs.px4.io/main/en/
Autopilot and simulation stack for aerial robotics projects.
RobotPerf. https://arxiv.org/abs/2309.09212
Benchmarking reference for robot computing performance.