Section 47.8: PX4 To Hardware: SITL, HITL, Logs, And Flight-Test Evidence

"A drone is not airworthy because it flew once; it is airworthy because the same fault was interrogated in SITL, in HITL, and in the logs."

A Systems-Minded Embodied AI Agent
Big Picture

The PX4-to-hardware path is not one test, it is an evidence ladder: SITL, HITL, estimator health checks, parameter review, controlled envelope expansion, and post-flight log analysis. A drone is not ready because it completed one route once. It is ready when the same fault model has been interrogated at every stage.

PX4 To Hardware: SITL, HITL, Logs, And Flight-Test Evidence conceptual illustration
Figure 47.8.1: A field-facing mental model for aerial robotics deployment. The illustration connects sensing, state estimation, planning, control, safety, and evidence logging across the full flight-readiness path.

Deployment Is A Sequence Of Claims

A PX4-based drone stack usually makes at least six distinct claims. First, the mission logic produces commands in the right frame and units. Second, the estimator keeps a believable state in the intended operating domain. Third, the low-level controller can track the commanded motion without persistent saturation. Fourth, the communication path between companion computer and flight controller meets the timing budget. Fifth, the safety monitor triggers the intended fallback. Sixth, the logs and replay tools are rich enough to explain a failure after the fact.

Those claims should be promoted stage by stage. SITL is where you catch frame mistakes, route logic bugs, and bad assumptions about the controller interface. HITL is where timing, estimator plumbing, and real flight-controller behavior become visible. Tethered or guarded hover tests are where vibration, battery sag, magnetic disturbance, prop wash, and sensor placement begin to rewrite the story.

Promotion Rule

Do not promote a drone stack to the next stage because the previous stage "looked good." Promote it only when the previous stage produced the exact evidence artifact that the next stage would need if it went wrong.

Flight-Readiness Mathematics

The most useful equations in deployment work are not fancy control laws. They are the diagnostics that tell you whether the stack still deserves trust. Two examples appear in almost every PX4 debugging session:

$$r_k = z_k - h(\hat x_k^-), \qquad a_y = v^2 \kappa.$$

The innovation residual $r_k$ says whether the estimator's belief and the incoming sensor measurement still agree. The lateral acceleration relation $a_y = v^2 \kappa$ says whether a planned turn is physically credible for the airframe and speed envelope. When a VIO-assisted mission drifts, or a trajectory asks for more curvature than the aircraft can safely track, these simple quantities fail before the mission-level score fails.

A practical flight-readiness gate can therefore be written as a vector of measurable conditions:

$$g = [e_{\mathrm{track}}, r_{\mathrm{ekf}}, s_{\mathrm{act}}, \ell_{\mathrm{latency}}, f_{\mathrm{failsafe}}].$$

The gate passes only if tracking error stays bounded, estimator residuals stay healthy, actuator saturation stays rare, link latency remains inside the offboard budget, and failsafes either do not trigger or trigger exactly as designed during test scenarios.

PX4 hardware promotion ladder A staged diagram connecting SITL, HITL, guarded hover, mission envelope expansion, and post-flight evidence review. SITL frames, logic route tests HITL timing, I/O real FCU Guarded hover vibration, EKF failsafes Envelope expansion wind, speed, payload, route Post-flight review logs, params, replay cases Promotion artifact: config, parameter diff, estimator health traces, actuator saturation, failsafe outcome, and replayable mission log Each stage inherits the exact metrics of the previous one and adds a new failure surface instead of replacing the evaluation.
Figure 47.8.2: A serious PX4 deployment workflow promotes the same mission through increasingly realistic stages while preserving metrics and replay artifacts.
Algorithm: PX4 promotion ladder
  1. Freeze the mission card: frames, units, setpoint interface, geofence, battery reserve, and emergency behaviors.
  2. Run SITL until route logic, offboard mode transitions, and frame conventions are clean under scripted perturbations.
  3. Move to HITL and measure command latency, estimator update timing, and mode transitions on real hardware.
  4. Run guarded hover or a tethered test and inspect vibration, innovation residuals, thrust saturation, and failsafe triggers.
  5. Expand the envelope gradually across wind, payload, speed, and route complexity, saving one replay artifact per failure class.

Practical Stack And What Each Tool Proves

The practical tool stack for this section is: PX4, QGroundControl, MAVLink, MAVSDK, ROS 2 uXRCE-DDS, Flight Review or Data Comets, Gazebo. The point is not to name a fashionable stack. The point is to assign each tool a job in the evidence ladder: PX4 exposes controller modes and failsafes, QGroundControl exposes parameters and health checks, MAVLink and MAVSDK expose command and telemetry contracts, ROS 2 exposes companion-computer timing, and the log-analysis tools expose what the vehicle actually believed and did.

Aerial hardware evidence checklist
StageMain questionEvidence to save
SITLAre frames, commands, and mission logic correct?Mission script, simulator seed, route outcomes, and offboard mode traces.
HITLDoes the real flight controller preserve timing and mode behavior?Mode transitions, command latency, estimator health, and parameter snapshot.
Guarded hoverCan the vehicle remain stable with the real airframe, sensors, and vibration?Innovation residuals, vibration metrics, actuator saturation, and failsafe results.
Envelope expansionWhich wind, payload, and route conditions remain inside the safe operating envelope?Per-flight envelopes, disturbance labels, recovery traces, and blocked conditions.
Post-flight reviewCan the team explain every anomaly and turn it into a reusable test?Annotated log review, replay case, mitigation note, and promotion decision.
Failure Modes To Test

Stress the system with frame-sign errors, estimator resets, vibration, magnetometer interference, motor imbalance, battery sag, payload shift, wind gusts, stale maps, and companion-computer latency. These are not rare corner cases. They are the normal reasons a beautiful simulation result becomes an unsafe aircraft.

Practical Example

A warehouse-inspection drone may pass SITL and HITL, then fail its first guarded hover because VIO timestamps lag just enough to produce innovation spikes during yaw motion. The right response is not "the model failed." The right response is to save the log, pin the failure to timing plus estimator fusion, and convert it into a permanent gate before the next flight.

Code And Evidence

The implementation below illustrates how to store one promotion decision as a compact artifact. Code Fragment 1 is intentionally small so that the structure, not the syntax, stays memorable.

# Build one promotion record for a PX4 hardware-readiness gate.
# The same schema should survive SITL, HITL, and guarded-flight stages.
from dataclasses import dataclass, asdict

@dataclass
class FlightGate:
    stage: str
    mean_tracking_error_m: float
    innovation_ratio: float
    command_latency_ms: int
    actuator_saturation_pct: float
    failsafe: str
    decision: str

    def as_row(self) -> dict[str, object]:
        return asdict(self)

gate = FlightGate(
    stage="guarded_hover",
    mean_tracking_error_m=0.18,
    innovation_ratio=0.74,
    command_latency_ms=32,
    actuator_saturation_pct=7.5,
    failsafe="not_triggered",
    decision="promote_to_low_speed_route_test",
)
print(gate.as_row())
{'stage': 'guarded_hover', 'mean_tracking_error_m': 0.18, 'innovation_ratio': 0.74, 'command_latency_ms': 32, 'actuator_saturation_pct': 7.5, 'failsafe': 'not_triggered', 'decision': 'promote_to_low_speed_route_test'}
Code Fragment 1: This promotion record captures the quantities that usually decide whether a PX4-based drone should advance to the next hardware stage. The output keeps estimator health, latency, saturation, and failsafe behavior in one artifact, which is exactly what a later incident review needs if the next mission goes wrong.

Expected output: the printed dictionary should make it possible to explain the promotion decision without opening another notebook or guessing which stage produced the numbers. If the record lacks stage name, estimator health, latency, or the explicit decision, it is too weak to support real hardware progression.

Library Shortcut

Use PX4 for controller modes and failsafes, QGroundControl for parameter management, MAVSDK for scripted missions, ROS 2 uXRCE-DDS for companion-computer integration, and Flight Review or Data Comets for log analysis. The point of the shortcut is not fewer lines of code, it is fewer silent assumptions between the planner and the propellers.

Recipe For Builders

  1. Freeze the mission manifest before hardware testing: airframe, payload, sensor suite, command interface, reserve battery, geofence, and abort conditions.
  2. Run SITL with the same route and metric script that you will use later on real hardware.
  3. Move to HITL only after command frames, mode transitions, and estimator sources are all explicit and reproducible.
  4. Use guarded hover and low-speed envelope expansion to test vibration, innovation residuals, saturation, and failsafes under controlled disturbances.
  5. Turn every anomaly into a replay case, then decide promotion or rollback from the evidence artifact rather than from team confidence.
Memory Hook

A drone stack is not "almost ready" when the route looks good. It is ready only when the next failure already has a log format, a replay path, and a blocked-promotion rule waiting for it.

Self Check

Can you state which one artifact would convince you to promote a PX4 mission from guarded hover to route flight, and which one artifact would force a rollback?

Research Frontier

Aerial autonomy research is moving toward larger learned mission models and richer simulation, but the hard unsolved problem is still disciplined promotion to hardware under uncertainty. The frontier contribution is often not a better policy alone, it is a better evidence loop linking learning, estimation, control, and flight readiness.

Key Takeaway

PX4 To Hardware: SITL, HITL, Logs, And Flight-Test Evidence earns its place in the book because it teaches the missing middle between a simulation result and a safe aircraft. That middle is where embodied AI becomes engineering.

Exercise 47.8.1

Create a flight-readiness package for one inspection mission: SITL result, HITL checklist, parameter diff, estimator-health plot, one failsafe test, and one replayable anomaly. End with a written promotion or rollback decision that cites the evidence directly.

Section References

PX4 Autopilot user guide. https://docs.px4.io/main/en/index

Official PX4 documentation for flight modes, simulation, configuration, estimators, and hardware bring-up.

PX4 companion-computer and ROS 2 guides. https://docs.px4.io/main/en/companion_computer/

Current official reference for companion-computer integration, ROS 2 routing, and offboard interfaces.

PX4 visual inertial odometry. https://docs.px4.io/main/en/computer_vision/visual_inertial_odometry

Official PX4 reference for GPS-denied VIO pipelines and estimator integration.

PX4 flight log analysis. https://docs.px4.io/main/en/log/flight_log_analysis

Official PX4 documentation for Flight Review and related log-analysis workflows.

MAVSDK. https://mavsdk.mavlink.io/main/en/

Programmatic mission control, telemetry, and system-state access for PX4-class vehicles.

EASA Specific Operations Risk Assessment, SORA. https://www.easa.europa.eu/en/domains/drones-air-mobility/operating-drone/specific-category-civil-drones/specific-operations-risk-assessment-sora

Operational risk framework that helps connect drone mission design to safety case obligations.