Section 7.3: PID control, intuition and tuning

A Careful Control Loop

This section develops the technical contract for PID control, intuition and tuning into a usable mental model. First we define the object of study, then we connect it to the agent loop, then we test it with a compact implementation.

The key question in PID control, intuition and tuning is practical: what must the agent know, what can it observe, what action is available, and what evidence shows that the action worked under the stated conditions?

Theory

For PID control, intuition and tuning, the practical design rule is to make the interface inspectable before optimization begins: inputs, outputs, units, latency, bounds, and failure labels should all be visible in the saved artifact.

PID control is useful because it gives three readable levers before a learned policy enters the loop. The proportional term reacts to the current error, the integral term removes persistent bias, and the derivative term resists fast changes that would otherwise overshoot. A tuning pass should change one lever at a time and record rise time, overshoot, settling time, steady-state error, saturation events, and noise sensitivity.

Worked Example

PID has three classic failure modes, and each one is reproducible in a few lines on the same 1D mass. Code Fragment 7.3.1 exposes all three: integral windup (a large \(K_i\) keeps charging while the actuator saturates, then overshoots), derivative kick (a step change in the setpoint makes \(\dot e\) spike, so a derivative computed on the error fires a huge first command), and derivative noise (differentiating a noisy measurement amplifies the noise). The fixes are an anti-windup clamp and computing the derivative on the measured output rather than on the error.

import numpy as np

m, dt, T = 1.0, 0.02, 5.0
n = int(T / dt)
u_max = 6.0

def run(kp, ki, kd, anti_windup=True, deriv_on_meas=False, ref=1.0):
    x = v = ei = 0.0
    e_prev = y_prev = 0.0     # e_prev = 0 so the setpoint step shows the kick
    overshoot, first_cmd = 0.0, None
    for k in range(n):
        y = x                                  # measured position
        e = ref - y
        ei_t = ei + e * dt
        # Derivative on measurement avoids the spike when the setpoint jumps.
        de = -(y - y_prev) / dt if deriv_on_meas else (e - e_prev) / dt
        u_unsat = kp * e + ki * ei_t + kd * de
        u = float(np.clip(u_unsat, -u_max, u_max))
        if (not anti_windup) or (u == u_unsat):    # anti-windup clamp
            ei = ei_t
        if first_cmd is None:
            first_cmd = u_unsat
        x += v * dt; v += (u / m) * dt
        overshoot = max(overshoot, x - ref)
        e_prev, y_prev = e, y
    return x, overshoot, first_cmd

xf, ov, _ = run(10, 3, 5)
print(f"tuned         : final={xf:.3f}  overshoot={ov:.3f}")
_, ov_off, _ = run(10, 30, 5, anti_windup=False)
_, ov_on,  _ = run(10, 30, 5, anti_windup=True)
print(f"windup off/on : overshoot {ov_off:.3f} -> {ov_on:.3f} with anti-windup clamp")
*_, kick_e = run(10, 3, 5, deriv_on_meas=False)
*_, kick_m = run(10, 3, 5, deriv_on_meas=True)
print(f"setpoint step : first command {kick_e:.1f} (deriv on error) vs {kick_m:.1f} (deriv on measurement)")

Practical Recipe

1. Write the observation, action, and success metric before choosing a model.
2. Build a baseline that is simple enough to debug by inspection.
3. Add the library implementation only after the baseline behavior is understood.
4. Record failures as structured cases: perception error, state error, planning error, control error, or evaluation error.
5. Run at least one perturbation test before trusting the result.

Production Pattern

PID control, intuition and tuning sits inside the Part II robotics contract: geometry defines where things are, kinematics defines what motion is possible, dynamics defines what motion costs, control defines how errors are corrected, and sensing defines what the agent can know on time.

Tune PID by reading proportional, integral, and derivative effects separately before combining them. This makes the section useful to students, builders, and researchers at the same time: the idea has an intuitive role, a formal interface, a runnable check, and a failure mode that can be reproduced.

Use this recipe when turning PID control, intuition and tuning into code, a simulator experiment, or a robot diagnostic. The point is not to use every library. The point is to keep the hand-built baseline and the maintained-tool path comparable.

1. Write the control objective, measured state, actuator command, update rate, and saturation policy.
2. Run a step-response test before adding learning, with overshoot, settling time, and steady-state error logged.
3. Compare the hand controller with python-control, CasADi, Drake, do-mpc, or ROS 2 control on the same plant model.
4. Record latency, missed deadlines, saturation events, constraint violations, and recovery actions.
5. Only compare controllers and policies when they share sensors, action limits, disturbance tests, and safety checks.

A learned policy can hide an unstable PID inner loop until the disturbance changes. Check update rate, delay, actuator saturation, integral windup, derivative noise, and fallback behavior before scaling training. For this section, first reproduce one tiny PID update by hand, then rerun it through python-control or the robot controller stack. If the two disagree, inspect sign convention, sample time, derivative filtering, and anti-windup behavior before changing the model.

# Compute one PID update from current error, accumulated error, and error slope.
# The sign of the derivative term shows whether the controller is damping the motion.
error_prev = 0.6
error_now = 0.4
integral_error = 1.2
dt = 0.1
kp, ki, kd = 2.0, 0.5, 0.1

derivative = (error_now - error_prev) / dt
command = kp * error_now + ki * integral_error + kd * derivative
print(f"derivative={derivative:.1f}, command={command:.1f}")