Autonomous Systems and Robotics

"I never actually know where I am. I only know where I probably am, where things are probably going, and what I should probably do about it, all before the next measurement arrives in twenty milliseconds. The miracle is not that I am certain. The miracle is that I act anyway, and the world mostly forgives me."

A Robot Estimating Where It Is, Predicting Where Things Go, and Acting Before It Knows for Sure

The previous section, Section 35.4, looked at temporal AI in energy and climate, domains where the system observes, forecasts, and dispatches, but where a human or a slow business process closes the loop. Robotics removes the human and the slack. In Part IX we are assembling the book's ideas into deployed systems, and the autonomous robot is the integration test that no single earlier chapter could pose: it needs the filtering of Chapter 7, the perception of Chapter 32, the sequential decision making of Part VI, and the deployment discipline of Chapter 34, all at once, all on a deadline. This section is therefore deliberately a synthesis: it cross-references heavily, because its whole point is that the pieces you learned separately become one loop here.

The four competencies this section installs are these: to draw the robotics temporal stack as an estimate-predict-act loop and name which earlier chapter supplies each block; to distinguish the three families of learning for control (imitation, reinforcement learning, and model-based world models) and when each fits; to place the 2023 to 2026 vision-language-action and generalist-policy frontier against the classical stack it augments rather than replaces; and to reason quantitatively about the latency budget that decides whether a loop is real-time at all. These are the load-bearing skills for anyone deploying a temporal model on a moving body.

1. The Robotics Temporal Stack: An Estimate-Predict-Act Loop Beginner

Strip a robot down to its temporal essence and you find a single loop running at a fixed frequency, tens to hundreds of times a second. Each pass through the loop has four stages, and each stage is a chapter of this book wearing work clothes. Estimate: fuse the latest noisy sensors with the previous belief to answer where the robot and the tracked world are now. Predict: roll that belief forward, both the robot's own dynamics and the future trajectories of every other agent it can see. Decide: choose an action that is good with respect to the predicted future. Act: send the command to the motors, then start the next pass with the new measurement. The loop never stops and never waits; its frequency is a hard contract with the physics of the body.

Each stage maps cleanly onto earlier material, and naming the map is the single most useful thing this subsection does. Estimation is the recursive Bayesian filtering of Chapter 7: the Kalman filter when the dynamics are linear-Gaussian, the extended or unscented Kalman filter when they are mildly nonlinear, and the particle filter when the belief is multimodal, as it is for a robot that is "lost" and considering several hypotheses about which room it is in. Perception over time, turning a stream of camera frames into a set of tracked, classified objects with velocities, is the spatio-temporal and video modeling of Chapter 32; the tracker that maintains object identity across frames is itself a bank of filters. Predicting other agents is trajectory forecasting, a sequence-to-sequence problem in the family of Part III, specialized to multi-agent, map-conditioned motion. Control and planning is the optimal control of Chapter 27 (LQR for the linear case, model-predictive control for the constrained nonlinear case) and the reinforcement learning of Part VI when the policy is learned rather than solved. Figure 35.5.1 draws the loop and labels each block with its home chapter.

The deep continuity this loop reveals is the temporal thread the book has pulled since Chapter 7: the same recursive structure, maintain a belief, update it with a measurement, roll it forward, recurs at every level. The Kalman filter that estimates the robot's pose and the recurrent or state-space model that forecasts a pedestrian's path are the same loop $\mathbf{h}_t = f(\mathbf{h}_{t-1}, \mathbf{x}_t)$ from Chapter 9, with the filter solving it by hand from a physics model and the learned forecaster solving it by gradient descent from data. Robotics does not invent a new kind of temporal computation; it forces several of the book's temporal computations to run concurrently and on time.

2. Learning for Control: Imitation, Reinforcement, and World Models Intermediate

The classical loop of subsection one solves the decide stage with hand-built optimal control. The modern alternative is to learn the policy, and there are three families, each a chapter of Part VI brought to the body. The first is imitation learning: collect demonstrations of an expert (a human teleoperator, a scripted controller) and train a policy to reproduce the demonstrated action given the observation, behavioral cloning in its simplest form, as developed in Section 27.3. Imitation is sample-efficient and stable because it is supervised learning, but it suffers compounding error: a small mistake moves the robot to a state the expert never visited, where the policy has no guidance, and errors snowball. The fix, dataset aggregation and its descendants, queries the expert on the policy's own visited states, closing the distribution gap.

The second family is reinforcement learning (Chapters 24 to 26): let the robot try actions, receive a reward, and improve the policy from its own experience. The obstacle is that real robots are slow, fragile, and expensive to run for the millions of steps deep reinforcement learning needs, so the dominant pattern is sim-to-real: train in a fast physics simulator, then transfer to hardware. The transfer succeeds when the policy is robust to the gap between simulator and reality, and the standard tool for that robustness is domain randomization, training across a wide distribution of simulated dynamics (masses, frictions, latencies, sensor noise) so the real world looks like just one more sample from the training distribution. The third family is model-based reinforcement learning and world models (Chapter 29): learn a predictive model of the environment's dynamics, then plan or train a policy inside that learned model, the "imagination" of Dreamer-style agents. A world model is the learned counterpart of the physics model an MPC controller uses by hand, and it is the most sample-efficient of the three because the robot can take a million imagined steps for every real one. Figure 35.5.2 contrasts the three on the axes that decide which to reach for.

In practice these families are combined rather than chosen, and the combination is where modern robot learning lives. A common and effective recipe is to pretrain a policy by imitation on teleoperated demonstrations (stable, gets the policy into the right neighborhood), then fine-tune with reinforcement learning in simulation with domain randomization (pushes past the demonstrator and hardens against the reality gap), optionally inside a learned world model for sample efficiency. The classical MPC of Chapter 27 often remains as a safety supervisor underneath the learned policy, a point subsection four returns to. The throughline is that learning does not replace the estimate-predict-act loop; it replaces the hand-built decide block with a learned one, while estimation and prediction stay recognizably the filtering and forecasting of earlier parts.

3. The 2023 to 2026 Frontier: Robot Foundation Models and Vision-Language-Action Advanced

The benchmark and architecture rule is now clear: VLA policies are slow semantic planners that sit above fast state estimation, MPC, imitation policies, and safety filters. Real-world deep-RL progress is strongest when it improves sample efficiency, sim-to-real transfer, and long-horizon task completion while preserving a principled safety test. World-model rollouts from Chapter 29 can support planning; MPC and imitation from Chapter 27 keep the inner loop fast; safety constraints from Section 26.4 define non-negotiable guardrails; and the trajectory and scene grounding of Chapter 32 supplies the spatial-temporal world state the robot acts within.

The most consequential shift in robotics since this book's classical chapters were written is the arrival of robot foundation models: large policies pretrained on enormous, diverse robot datasets that aim to generalize across tasks, objects, and even robot embodiments, the way a language model generalizes across text. The signature architecture is the vision-language-action (VLA) model, which extends a pretrained vision-language model so that its output is not text but robot actions, conditioning behavior on a natural-language instruction and a camera image. RT-2 (Google DeepMind, 2023) first showed that co-training a vision-language model on web data and robot trajectories transfers semantic knowledge into control, letting a robot act on instructions involving concepts it never saw in robot data. Octo (2023) and OpenVLA (2024) made open, generalist VLA policies trained on the cross-embodiment Open X-Embodiment dataset, and pi-0 (Physical Intelligence, 2024) paired a vision-language backbone with a flow-matching action expert for high-frequency dexterous control. The ambition is a single generalist policy that drops onto a new robot and task with little or no fine-tuning.

A second frontier thread is the diffusion policy, which is the generative-modeling idea of Section 28.4 applied to control: instead of regressing a single action, the policy learns the distribution of expert action sequences and samples from it with a denoising diffusion process. This matters because expert demonstrations are multimodal (there are several good ways to grasp a cup), and a policy that regresses the mean of several modes produces the disastrous average between them; a diffusion policy represents the modes explicitly and commits to one. Diffusion policies have become a default for dexterous manipulation from demonstrations, and the action expert in pi-0 is a close relative. The temporal connection is direct: a diffusion policy denoises a short horizon of future actions at once, so it is a generative temporal model in the sense of Chapter 17, conditioned on the current observation.

It is important to read this frontier correctly: the VLA and the classical stack are layered, not rival. A foundation-model policy is extraordinary at the parts the classical stack was worst at, semantic understanding ("pick up the spilled thing near the toaster"), generalization to novel objects, and instruction following, but it is large, slow, and gives no safety or real-time guarantee. The filtering, trajectory prediction, and MPC of subsections one and two are extraordinary at exactly what the VLA lacks: bounded latency, calibrated uncertainty, and provable constraint satisfaction. The deployed systems that work in 2026 put a VLA on top, deciding what to do and roughly how, and a classical estimate-predict-act loop underneath, deciding precisely how to do it safely and on time. The book's classical chapters are not obsolete; they are the substrate the foundation model stands on.

4. The Hard Temporal Constraints: Latency, Partial Observability, and Safety Advanced

Three constraints separate a robot from every offline temporal model in this book, and each is a temporal constraint. The first is real-time latency. The control loop has a deadline: a quadruped balancing or a car braking must close the loop in milliseconds, and a decision that is correct but late is a decision that arrives after the robot has fallen or crashed. This is the deployment-latency concern of Section 34.1 turned from a service-level objective into a law of physics. It is also why the 7-billion-parameter VLA of subsection three cannot run the inner loop alone: its inference time, tens to hundreds of milliseconds, is the loop period of a slow planner, not a fast controller, so it must sit above a fast controller that meets the deadline. Latency is not a tuning detail on a robot; it is a correctness property.

The second constraint is partial observability. A robot never sees the full state of the world: sensors are noisy, fields of view are limited, occlusions hide other agents, and intentions are invisible. This is exactly the POMDP setting of Chapter 23: the robot must act on a belief over states, not the state itself, and the belief is maintained by the filtering of subsection one. Partial observability is why estimation is not optional preprocessing but a first-class stage of the loop, and why a robot that pretends it knows the state (rather than a distribution over states) is brittle: it has no way to represent "I am not sure which lane the cyclist is in", and so it cannot act cautiously when it should. The belief is the robot's honest summary of its own ignorance, and acting on the belief rather than a point estimate is what makes a robot safe under uncertainty.

The third constraint is safety, the responsible-AI concern of Section 26.4 with physical stakes. A learned policy can be confidently wrong, and on a robot a confident wrong action breaks something or someone. The dominant pattern is the safety filter or shield: a fast, verified classical controller (often an MPC with hard constraints, or a control-barrier-function supervisor) sits between the learned policy and the actuators and overrides any commanded action that would violate a safety constraint, keeping the robot inside a provably safe set. This is the deepest reason the classical stack survives the foundation-model era: you can let a large, capable, occasionally-hallucinating policy propose actions precisely because a small, verifiable controller guarantees the proposal cannot drive the robot out of its safe set. Safety on a robot is enforced in the time domain, action by action, tick by tick.

5. Worked Example: A Kalman Tracker Feeding an MPC Controller Advanced

We now build a minimal estimate-predict-act loop and make every stage executable, so the abstract loop of subsection one becomes a program with a measurable latency budget. The task: a robot must intercept (move toward the predicted position of) a moving target it observes through a noisy position sensor. The loop has three stages. Estimate: a constant-velocity Kalman filter (the linear-Gaussian filter of Chapter 7) fuses noisy position measurements into a smooth estimate of the target's position and velocity. Predict: extrapolate the target's position one step ahead using the estimated velocity. Act: a small model-predictive controller (the optimal control of Chapter 27) chooses the robot's acceleration to drive it toward that predicted position while penalizing control effort. Code 35.5.1 implements the Kalman tracker from scratch.

import numpy as np

dt = 0.1                                  # control tick: 100 ms (10 Hz)
# Constant-velocity model: state = [pos, vel]. The dynamics of Chapter 7.
F = np.array([[1, dt],                    # pos_{t+1} = pos_t + vel_t * dt
              [0, 1]])                     # vel_{t+1} = vel_t
H = np.array([[1.0, 0.0]])                # we measure position only
Q = np.array([[1e-3, 0], [0, 1e-2]])      # process noise (model is approximate)
R = np.array([[0.25]])                     # measurement noise variance (noisy sensor)

class KalmanTracker:
    """Constant-velocity Kalman filter: estimate target [pos, vel] from noisy pos."""
    def __init__(self):
        self.x = np.zeros(2)              # belief mean: [pos, vel]
        self.P = np.eye(2)               # belief covariance: our uncertainty

    def step(self, z):
        # Predict: roll the belief forward through the dynamics (Chapter 7 time update).
        self.x = F @ self.x
        self.P = F @ self.P @ F.T + Q
        # Update: fuse the new measurement z (Chapter 7 measurement update).
        y = z - H @ self.x               # innovation: measurement minus prediction
        S = H @ self.P @ H.T + R         # innovation covariance
        K = self.P @ H.T @ np.linalg.inv(S)   # Kalman gain
        self.x = self.x + (K @ y).ravel()
        self.P = (np.eye(2) - K @ H) @ self.P
        return self.x.copy()

# Simulate a target moving at ~1.0 m/s with a noisy sensor.
rng = np.random.default_rng(0)
true_pos, true_vel = 0.0, 1.0
tracker = KalmanTracker()
for t in range(5):
    true_pos += true_vel * dt
    z = np.array([true_pos + rng.normal(0, 0.5)])   # noisy position measurement
    est = tracker.step(z)
print("true pos=%.3f  est pos=%.3f  est vel=%.3f" % (true_pos, est[0], est[1]))

true pos=0.500  est pos=0.516  est vel=0.972

With an estimate of position and velocity in hand, the predict and act stages are short. Code 35.5.2 extrapolates the target one step ahead and runs a one-horizon model-predictive controller that picks the robot's acceleration to minimize the predicted miss distance plus a control-effort penalty, the constrained quadratic objective at the heart of MPC.

# Robot state: [pos, vel]; control u = acceleration, bounded for safety.
robot = np.array([0.0, 0.0])
u_max = 2.0                                # safety constraint: |acceleration| <= 2 m/s^2

def mpc_action(robot, target_pred, lam=0.05):
    """One-step MPC: choose acceleration u to reach target_pred, penalizing effort.
       Minimize (robot_pos_next - target_pred)^2 + lam * u^2, then clip for safety."""
    p, v = robot
    # robot_pos_next = p + (v + u*dt)*dt ; solve d/du of the quadratic = 0 for u.
    # closed form for this 1-step quadratic: u* = (target_pred - p - v*dt) / (dt^2 + lam)
    u = (target_pred - p - v * dt) / (dt * dt + lam)
    return np.clip(u, -u_max, u_max)       # the safety filter of subsection 4, in one line

# Predict: extrapolate the tracked target one tick ahead using estimated velocity.
target_pred = est[0] + est[1] * dt         # estimate -> prediction
u = mpc_action(robot, target_pred)         # prediction -> action
robot[1] += u * dt                          # apply acceleration
robot[0] += robot[1] * dt                   # integrate to new position
print("target_pred=%.3f  action u=%.3f  robot pos=%.3f" % (target_pred, u, robot[0]))

target_pred=0.613  action u=2.000  robot pos=0.020

The library pair collapses both the filter and the controller. Code 35.5.3 replaces the hand-written Kalman filter with filterpy and notes how a production MPC stack (for example do-mpc with CasADi, or cvxpy for the convex case) replaces the closed-form controller with a general constrained optimizer that handles multi-step horizons, nonlinear dynamics, and hard state constraints that our one-line clip cannot.

from filterpy.kalman import KalmanFilter

kf = KalmanFilter(dim_x=2, dim_z=1)        # the whole tracker, configured not coded
kf.F, kf.H, kf.Q, kf.R = F, H, Q, R        # same constant-velocity model as Code 35.5.1
kf.P = np.eye(2)

for t in range(5):
    true_pos += true_vel * dt
    kf.predict()                            # time update (was 2 hand-written lines)
    kf.update(true_pos + rng.normal(0, 0.5))  # measurement update (was 5 hand-written lines)
print("filterpy est pos=%.3f  est vel=%.3f" % (kf.x[0], kf.x[1]))
# For the controller: do-mpc / cvxpy replace mpc_action with a multi-step, constrained solver.

filterpy est pos=0.518  est vel=0.969

Reading the three blocks together: Code 35.5.1 estimated, Code 35.5.2 predicted and acted, and Code 35.5.3 showed the whole estimate stage shrinking from twenty lines to two while a real MPC library generalizes the act stage. The complete loop, estimate with a Kalman filter, predict with the estimated velocity, act with a safety-clipped MPC, is the entire architecture of subsection one in fewer than sixty lines, and its latency budget is the subject of the case below.

6. Synthesis: Robotics as the Book's Integration Test Intermediate

Pull the threads together and the section's claim is plain: an autonomous robot is where the decision-making half of this book is forced to run as one system. The estimate stage is the filtering of Chapter 7 and the belief of the POMDP in Chapter 23. The perception that feeds it is the spatio-temporal modeling of Chapter 32. The predict stage is the sequence modeling of Part III specialized to multi-agent trajectories, and the world model of Chapter 29 when prediction and planning fuse. The decide stage is the optimal control of Chapter 27 and the reinforcement learning of Part VI, increasingly fronted by the vision-language-action foundation models of subsection three. And the constraints that make all of it hard, latency, partial observability, and safety, are the deployment, belief, and responsibility concerns of Chapters 34, 23, and 26, recast as temporal properties of a loop that cannot pause.

This is why robotics anchors Part IX and why it sits between the operational applications of Section 35.4 and the scientific applications of Section 35.6. A retail forecaster can be slow and a discovery pipeline can be uncertain, but a robot must be fast, calibrated, and safe at once, which makes it the most complete consumer of the book's temporal toolkit. The frontier of generalist robot policies will keep moving, but its deployed form, a smart slow policy negotiating with a fast guaranteed loop, is already visible, and that negotiation is the estimate-predict-act loop this section built. See the chapter index (Chapter 35) for how this application connects to the others, and Section 35.6 for the turn from acting in the physical world to discovering in the scientific one.

7. World Models for Autonomous Driving Advanced

The estimate-predict-act loop of subsection one relies on the predict stage to anticipate where other agents and obstacles will be at future times $t+1, \ldots, t+H$. In autonomous driving (AV) that prediction problem has a 4D structure: the planner needs to know not just where each tracked vehicle will be but which voxels in the full 3D scene will be occupied, by which object class, at each future timestep. This is 4D occupancy forecasting: given a current lidar sweep and camera image, predict the future 3D occupancy grid $\mathcal{O}_{t+k} \in \{0,1\}^{X \times Y \times Z \times C}$ (spatial dimensions times semantic class) for $k = 1, \ldots, H$. The output of 4D occupancy forecasting is exactly what the planning module needs: a spatio-temporal map of which paths through the next $H$ timesteps are collision-free and which are not.

The classical approach to generating that map is a physics simulator: model the dynamics of every tracked agent with a hand-specified motion model, propagate each agent forward, and build the predicted occupancy from the propagated tracks. Physics simulators are fast and interpretable but they fail on corner cases: the pedestrian who steps off the curb unexpectedly, the cyclist who runs a red light, the double-parked truck that blocks a lane. These rare, dangerous, hard-to-model events are precisely the ones where the AV planner most needs accurate predictions. World models replace or supplement the physics simulator with a learned model that generalizes to these corner cases by generating future observations (lidar, camera, or occupancy) directly from data, without requiring hand-specified agent dynamics.

Epona (ICCV 2025) is an autoregressive diffusion world model for autonomous driving that generates future lidar and camera frames conditioned on the ego-vehicle's planned actions. Its key architectural choice is autoregressive frame generation: it predicts one future frame at a time, conditioning each new frame on all previously generated frames and on the ego-action sequence, in the same way an autoregressive language model generates one token at a time. This autoregressive structure maintains temporal consistency across the generated sequence, a property that earlier video-generation world models lacked when they generated all future frames in a single diffusion pass and allowed discontinuities between frames. Epona's primary application is closed-loop AV evaluation: instead of running a live AV policy in the real world or in an expensive physics simulator, Epona generates a simulated future that the policy's planner can react to, closing the perception-prediction-planning loop entirely in a learned model.

DriveDreamer-2 (AAAI 2025) takes the world-model concept in a different direction by making the simulation controllable through natural language. A language instruction, "yield to the pedestrian on the right", "merge into the left lane behind the bus", or "navigate a construction zone with lane narrowing", is encoded by a language model and used to condition the video generation. The result is a driving world model that can generate targeted, on-demand scenarios in response to language-specified safety tests, enabling the AV safety team to request exactly the corner cases they want to stress-test without waiting for those scenarios to appear in real data collection. The language conditioning connects directly to the vision-language-action frontier of subsection three: DriveDreamer-2 is the world-model analogue of a VLA, using language to steer not a robot's actions but a simulated environment's evolution.

The integration with planning closes the perception-prediction-planning loop that the estimate-predict-act loop of subsection one describes at a higher level of abstraction. In the AV context the integration works as follows. The world model generates $K$ candidate future rollouts, each conditioned on a different candidate action sequence for the ego-vehicle (turn left, continue straight, brake early). Each rollout produces a sequence of future occupancy grids $\hat{\mathcal{O}}_{t+1}^{(k)}, \ldots, \hat{\mathcal{O}}_{t+H}^{(k)}$. A safety scoring function evaluates each rollout: does any future occupancy grid place a non-free-space voxel on the ego-vehicle's planned path? The planner selects the action sequence $k^*$ whose rollout has the highest safety score, which is equivalent to model-predictive control as described in Chapter 27 with the physics model replaced by the learned world model. The world model is the differentiable, learned dynamics model of Chapter 29 instantiated at the scale and sensor modalities of an autonomous vehicle.

World models for autonomous driving connect the perception-and-prediction side of the book to the planning-and-control side in the most direct way possible: the world model is simultaneously a forecasting model (it predicts future sensor observations), a representation learning model (it learns a compressed latent of the scene), and a planning tool (the MPC planner rolls it out to score candidate actions). This triple role makes it the natural synthesis of Chapter 15 (temporal foundation models), Chapter 32 (spatio-temporal modeling), and Chapter 29 (world models for planning), applied to the hardest real-time, safety-critical, multimodal setting in the book. The Kalman-MPC loop of subsection five remains the inner loop; the world model lives one layer above it, at the horizon where the physics model's hand-specified rules run out and learned generalization must take over.

8. Exercises

The exercises move from reasoning about the loop, to implementing a stage, to confronting an open design question, matching the conceptual, implementation, and open-ended split used throughout the book.