Chapter 30: Navigation and Path Planning | Building Embodied AI: From Perception to Autonomous Action

"An agent becomes interesting at the exact moment perception changes what it dares to do next."
A Patient Embodied AI Agent

Big Picture

Navigation and Path Planning turns perception into action-ready state. Navigation is where perception becomes a commitment. The agent has to choose a route, spend time, avoid contact, and still recover when the world refuses to match the map.

Remember This Chapter

The durable test is not whether a model looks impressive. The test is whether it improves a robot's next action while leaving a clear evidence trail for debugging.

Chapter Overview

Chapter 30 develops Navigation and Path Planning as a working piece of the embodied AI stack. It connects visual or spatial evidence to state estimates, action choices, visual servoing loops, timing budgets, and failure labels.

The chapter follows the right-tool rhythm used across the book: build the mechanism once, then move to maintained tools such as NetworkX, NumPy, ROS 2 Nav2, Habitat.

Prerequisites

Readers should be comfortable with Python, tensors, coordinate frames, sensor noise, and the perception-action loop. Useful refreshers appear in Chapter 4, Chapter 8, and Chapter 13.

Chapter Roadmap

30.1 Navigation as embodied intelligencenavigation combines perception, mapping, planning, control, timing, and recovery.
30.2 Graph search: BFS, Dijkstra, A*graph search converts a map into nodes, edges, costs, and a rule for expanding the next state.
30.3 Sampling-based planning: RRT, RRT*, PRMsampling-based planners search continuous spaces by growing trees or roadmaps through collision-free samples.
30.4 Local planning and obstacle avoidance (DWA, potential fields)local planners translate a global route into immediate velocity commands that respect dynamics and nearby obstacles.
30.5 Learned navigation policieslearned policies can map observations to actions, but they still need evaluation against classical planners and recovery cases.
30.6 Language- and image-goal navigationgoal-conditioned navigation turns words or images into a search target that must be grounded in perception and maps.
30.7 Field Navigation Under Degraded SensingNavigation under degraded sensing is a risk-managed control problem: the robot must reason about map confidence, local traversability, fallback policies, and when not to continue.

Tooling Note

This chapter uses the right-tool principle. The teaching baseline exposes units, frames, uncertainty, and logging. The shortcut stack uses maintained tools to handle optimized kernels, visualization, data formats, simulation hooks, and deployment interfaces.

Hands-On Lab: Build A Navigation and Path Planning Evidence Panel

Duration: about 75 minutesDifficulty: Intermediate

Objective

Build a small evidence panel that compares a hand-built baseline with a maintained tool workflow for this chapter.

What You'll Practice

Writing an observation, action, metric, and perturbation contract.
Building one inspectable baseline before using a library shortcut.
Logging success, failure labels, latency, and recovery behavior.
Explaining which result would change a robot action.

Setup

Use a Python environment with NumPy. Add chapter-specific tools only after the baseline manifest runs.

# Create a small local environment for the chapter lab.
python -m pip install numpy

Code Fragment 30.L1 installs NumPy for the chapter evidence manifest before heavier perception tools are added.

Steps

Step 1: Define The Contract

Write observation, action, metric, and perturbation fields for two sections.

Step 2: Run The Baseline Manifest

Create one comparable row per section, then fill realistic values from the section text.

# Start a Chapter 30 evidence manifest.
# Add one row per section and keep metrics construct matched.
sections = ['30.1', '30.2', '30.3', '30.4', '30.5', '30.6']
manifest = [
    {"section": s, "metric": "closed_loop_success", "perturbation": "occlusion_or_noise"}
    for s in sections
]
print(manifest[0])

Code Fragment 30.L2 creates `manifest` rows for Chapter 30, giving each section the same metric and perturbation fields.

Step 3: Add The Library Shortcut

Replace one baseline field with a maintained tool call, while keeping the output schema unchanged.

Step 4: Run One Perturbation

Add occlusion, noise, pose drift, map error, or goal ambiguity. Record whether the action changed.

Step 5: Write The Postmortem

Explain the strongest result, the most informative failure, and the next diagnostic test.

Expected Output

A table with one row per tested section, one baseline result, one shortcut result, one perturbation label, and one failure label.

Stretch Goals

Add a plot of metric versus perturbation strength.
Run the same manifest in a Habitat-style simulator or ROS 2 bag replay.
Export two failure cases with enough metadata to reproduce them later.

Complete Solution

# Start a Chapter 30 evidence manifest.
# Add one row per section and keep metrics construct matched.
sections = ['30.1', '30.2', '30.3', '30.4', '30.5', '30.6']
manifest = [
    {"section": s, "metric": "closed_loop_success", "perturbation": "occlusion_or_noise"}
    for s in sections
]
print(manifest[0])
for row in manifest:
    row["baseline_score"] = 0.72
    row["shortcut_score"] = 0.81
    row["failure_label"] = "perception_or_planning_interface"
print(manifest)

Code Fragment 30.L3 fills the manifest with comparable baseline and shortcut fields for a same-panel chapter lab.

Use this chapter as a complete teaching unit: concept, minimal implementation, library shortcut, diagnostic perturbation, and postmortem. The pattern prevents a perception model from being evaluated in isolation and never tested as part of the agent loop.

Chapter Tool Map

Tool or Library	Where It Pays Off
NetworkX	Use when it shortens the path from mechanism to reproducible embodied evidence.
NumPy	Use when it shortens the path from mechanism to reproducible embodied evidence.
ROS 2 Nav2	Use when it shortens the path from mechanism to reproducible embodied evidence.
Habitat	Use when it shortens the path from mechanism to reproducible embodied evidence.
PyTorch	Use when it shortens the path from mechanism to reproducible embodied evidence.
OpenCV	Use when it shortens the path from mechanism to reproducible embodied evidence.
Open3D	Use when it shortens the path from mechanism to reproducible embodied evidence.

Readiness Check

Before leaving the chapter, the reader should be able to state one theory claim, one implementation claim, one evaluation claim, and one realistic failure mode.

Teaching Takeaway

A strong chapter session ends with an artifact: a script, trace, simulator run, data card, map, or reproducible evaluation panel.

What's Next?

Start with Section 30.1: Navigation as embodied intelligence. After this chapter, continue to Chapter 31: Language-Guided Embodied Agents.

Bibliography & Further Reading

Foundational Papers, Tools, and References

LaValle, S. M.. "Planning Algorithms." Cambridge University Press, 2006. http://lavalle.pl/planning/

The classic open reference for graph search, sampling-based planning, and configuration spaces.

Macenski, S. et al.. "Robot Operating System 2: Design, architecture, and uses in the wild." Science Robotics, 2022. https://www.science.org/doi/10.1126/scirobotics.abm6074

Context for ROS 2 and the deployed robotics middleware surrounding navigation stacks.

Savva, M. et al.. "Habitat: A Platform for Embodied AI Research." ICCV, 2019. https://arxiv.org/abs/1904.01201

A common simulator reference for embodied navigation workflows and benchmark tasks.

ROS 2 Navigation. "Nav2 documentation." Project documentation. https://navigation.ros.org/

The practical reference for global planning, local control, costmaps, behavior trees, and recovery behaviors.