"I never see the thing I am trying to know. I see only its shadow, smeared by noise, arriving one frame at a time. So I keep a belief about the truth, a whole distribution, and I refuse to let it collapse into a single number however loudly any one measurement insists. Each observation nudges me, and I weigh it against what I already believe in exact proportion to how much I trust each: a confident sensor moves me far, a noisy one barely at all. Then I roll my belief forward through my model of how the world changes and wait for the next frame. Carry the state, update it, predict, repeat. I have been doing this since 1960, and the recurrent networks think they invented it."
A Belief Distribution Refusing to Collapse
Chapter Overview
Chapter 6 closed on a quiet revelation: the dynamic factor model treats its observed series as noisy projections of a few hidden drivers that evolve over time, and it is fit by a recursion that estimates those drivers as each observation arrives. That recursion has a name, the Kalman filter, and that hidden-driver picture has a name, the state-space model. This chapter makes the hidden state the central object of classical time series. A state-space model posits a latent state that evolves under its own dynamics and that we observe only through a noisy measurement, and the remarkable fact that organizes the whole chapter is that nearly every model of Part II is a state-space model in disguise. ARIMA, exponential smoothing, unobserved-components, and dynamic factor models all share one transition-and-observation form, which means one inference engine fits them all.
That engine is the Kalman filter, and it is one of the most consequential algorithms in the history of applied mathematics. For a linear-Gaussian state-space model it computes the exact posterior over the hidden state, optimally and recursively, in two steps that repeat forever: a predict step that rolls the belief forward through the dynamics, and an update step that folds in the new measurement weighted by the Kalman gain, a trust trade-off that leans on the model when the sensor is noisy and on the sensor when the model is uncertain. It is recursive Bayesian inference in closed form, it needs only the current state to summarize all of the past, and it produces, as a free by-product, the prediction-error likelihood that lets you fit the model's parameters by maximum likelihood. The chapter builds this filter from first principles and then runs it backward as the RTS smoother to refine past states in light of later data.
The world is rarely linear or Gaussian, so the chapter extends the filter outward in two directions. When the dynamics are nonlinear, the Extended Kalman Filter linearizes them with a Jacobian and the Unscented Kalman Filter propagates a small set of sigma points through the true nonlinearity, trading a derivative for a deterministic sampling scheme. When the posterior is genuinely non-Gaussian, multimodal, or skewed, the particle filter abandons the Gaussian assumption entirely and represents the belief with a weighted cloud of samples that it propagates, reweights, and resamples, a sequential Monte Carlo method that converges to the exact posterior as the cloud grows. The hidden Markov model is the discrete-state cousin of all of this: when the latent state is one of finitely many regimes rather than a continuous vector, the same predict-update recurrence becomes the forward algorithm, Viterbi decoding finds the single best path, and Baum-Welch learns the model, the tool of choice for regime-switching series and labeled-sequence problems.
The temporal thread is at its strongest in this chapter, because this is the chapter where the thread is anchored. Strip a filter to its skeleton and you find a recurrence: it carries a state, it updates that state with each observation, and it reads out a prediction. That is exactly the computation a recurrent neural network performs, except that the RNN learns its transition and readout from data instead of being handed them as known matrices, and so handles the nonlinear, non-Gaussian, hard-to-specify dynamics the classical filters strain against. The closing section makes this precise: the Kalman filter, the HMM forward pass, and the exponential smoother are all special cases of one recurrence, and that recurrence is the direct classical ancestor of the recurrent network of Chapter 10 and the structured state-space models of Chapter 13. Following the Part II rotation, the worked examples lean on a healthcare-flavored tracking problem, an irregularly observed vital sign whose true value must be inferred through measurement noise, with a finance regime-switching series exercising the discrete cousin.
Prerequisites
This chapter rests on two earlier ones and on a little linear algebra. From Chapter 3: Statistical Foundations you need stationarity, white noise, the autocovariance structure of a process, and basic Bayesian reasoning with Gaussians, because the Kalman filter is recursive Bayesian inference and its steady-state behavior is a stationarity statement about the filtered state. From Chapter 5: Univariate Forecasting Models you need ARIMA and exponential smoothing, since the chapter's organizing claim is that both are state-space models and the exponential smoother is a one-state Kalman filter; the lab and the bridge section assume you can read an ARIMA or ETS fit. Beyond the time-series prerequisites you need comfort with linear algebra and multivariate Gaussians: matrix multiplication, inversion, covariance matrices, and the fact that a linear map of a Gaussian is Gaussian, because the predict and update equations are exactly those operations written for a state vector and its covariance. Readers wanting a refresher on any of these will find the relevant chapters and the linear-algebra and probability appendices indexed in the Table of Contents.
If you keep one idea from this chapter, keep this: a filter is a recurrence, carry a state, update it with each observation, read out a prediction, and the Kalman filter does this optimally for linear-Gaussian systems while the RNN learns to do it for everything else. The state-space form unifies ARIMA, ETS, and factor models under one transition-and-observation equation, so one inference engine serves them all. The Kalman gain is the heart of the update: it weighs each new measurement against the current belief in exact proportion to their relative trust. When the world turns nonlinear you linearize (EKF) or sample (UKF); when it turns non-Gaussian you carry a particle cloud; when the state is discrete you run the HMM forward algorithm. Every one of these is the same predict-update loop, and that loop is the seed from which the recurrent networks and structured state-space models of Part III grow.
Chapter Roadmap
- 7.1 Hidden States and the State-Space Formulation The latent state as a compressed Markov summary of the past that is sufficient for the future, the linear-Gaussian transition and observation equations, the filtering, prediction, and smoothing inference tasks, and why ARIMA, ETS, unobserved-components, and dynamic factor models all share one state-space form, with a local-level model built from scratch and through statsmodels.
- 7.2 The Kalman Filter The recursive Bayesian view of linear-Gaussian filtering, the predict and update steps, the Kalman gain as a trust trade-off between model and measurement, minimum-variance optimality and the steady-state filter, the prediction-error log-likelihood for parameter estimation, and the RTS smoother, with a complete from-scratch filter checked against filterpy and statsmodels.
- 7.3 Extended and Unscented Kalman Filters Filtering nonlinear state-space models, why a Gaussian belief stops being Gaussian under a nonlinear map, the Jacobian-based Extended Kalman Filter, the sigma-point Unscented Kalman Filter and its tuning, the ensemble filter for high dimensions, and a worked pendulum-tracking comparison of the two approaches.
- 7.4 Particle Filters and Sequential Monte Carlo When Gaussian filters fail, represent the posterior with a weighted cloud of particles, importance sampling, the bootstrap filter, effective sample size and the resampling step that fights particle degeneracy, and a from-scratch implementation on a nonlinear, non-Gaussian state-space model.
- 7.5 Hidden Markov Models The discrete-state cousin of the Kalman filter, latent states with transition probabilities and emission distributions, the forward algorithm, Viterbi decoding, Baum-Welch learning, Gaussian and categorical emissions, and Markov-switching regimes, with a from-scratch then hmmlearn worked example on a regime-switching finance series.
- 7.6 Bridge: Filtering as Recurrent Inference The unifying argument that the Kalman filter, the HMM forward pass, and the exponential smoother are all special cases of one recurrence that carries a state forward, updates it with each observation, and reads out a prediction, and how that recurrence becomes the recurrent neural network, the structured state-space model, and the attention summary of Part III.
Once you have worked through the six sections, the Hands-On Lab below chains them into a single tracking study. Each lab step maps to a section, so the lab doubles as a review: by the time you finish you will have run the linear Kalman filter of Section 7.2, the Extended and Unscented filters of Section 7.3, and the particle filter of Section 7.4 on one nonlinear tracking problem, fit the hidden Markov model of Section 7.5 to a regime-switching variant, and sketched the same predict-update loop as the recurrent cell of Section 7.6.
Hands-On Lab: Track a Hidden State Four Ways
Objective
Take one nonlinear, noisy tracking problem and estimate its hidden state four different ways, then watch the same recurrence reappear as a neural cell. You will simulate a system whose true latent state evolves under mildly nonlinear dynamics and is seen only through a noisy nonlinear measurement, the kind of problem a bedside monitor faces when it must infer a true physiological value from a jittery sensor. First you will run a plain linear Kalman filter as a deliberately mis-specified baseline, linearizing the problem crudely so you can feel where the linear assumption hurts. Then you will run an Extended Kalman Filter that linearizes with a Jacobian and an Unscented Kalman Filter that propagates sigma points, and compare their tracking error and runtime. Then you will run a particle filter that carries a full cloud of samples and handles the non-Gaussian posterior the Gaussian filters approximate. With the continuous filters compared, you will switch to a regime-switching variant of the signal and fit a hidden Markov model that recovers the discrete hidden states behind the regimes. Finally you will write the Kalman predict-update step as a tiny recurrent cell, a hand-built RNN that carries a state and updates it with each observation, and see that the classical filter and the neural recurrence are the same shape. The lab is the chapter in miniature: one hidden state, five inference engines, one recurrence underneath them all.
What You'll Practice
- Running a linear Kalman filter as a baseline and reading its tracking error against the known truth, following Section 7.2.
- Implementing and comparing an Extended and an Unscented Kalman Filter on a nonlinear system, trading a Jacobian for sigma points, following Section 7.3.
- Running a particle filter with importance weighting and resampling, and watching effective sample size, following Section 7.4.
- Fitting a hidden Markov model to a regime-switching series and decoding the hidden states with Viterbi, following Section 7.5.
- Writing the predict-update loop as a minimal recurrent cell and seeing the filter-as-RNN equivalence, following Section 7.6.
Setup
You need filterpy for the Kalman, Extended Kalman, and Unscented Kalman filters and for a clean particle-filter scaffold, hmmlearn for the hidden Markov model, and NumPy for the from-scratch particle cloud and the recurrent cell. A small simulator for the nonlinear tracking problem and its regime-switching variant ships with the lab so it runs out of the box, and the bibliography links Labbe's filterpy companion text when you want to go deeper on any filter. Every step compares each estimator against the simulator's known true state, so accuracy is measured directly rather than inferred, and respects the leakage discipline of Chapter 2: a filter sees each observation only once, in order, never the future.
pip install filterpy hmmlearn numpy scipy matplotlibSteps
Step 1: Run a linear Kalman filter as the baseline
Start with the simplest estimator, a linear Kalman filter applied to a crudely linearized version of the problem, following Section 7.2. The predict step rolls the state and its covariance forward through the linear dynamics, and the update step folds in each measurement weighted by the Kalman gain. Record the root-mean-square tracking error against the simulator's true state; it sets the bar the nonlinear filters must beat and exposes exactly where the linear assumption breaks.
from filterpy.kalman import KalmanFilter
import numpy as np
def linear_baseline(z, F, H, Q, R, x0, P0):
kf = KalmanFilter(dim_x=len(x0), dim_z=1)
kf.F, kf.H, kf.Q, kf.R = F, H, Q, R # linearized dynamics and noise
kf.x, kf.P = x0, P0 # initial state and covariance
est = []
for zk in z:
kf.predict() # roll belief forward
kf.update(zk) # fold in the measurement via the Kalman gain
est.append(kf.x.copy())
return np.array(est) # filtered state path
Step 2: Run an Extended and an Unscented Kalman Filter
Bring the nonlinearity back and handle it two ways, following Section 7.3. The Extended Kalman Filter linearizes the true dynamics and measurement at the current estimate using their Jacobians, while the Unscented Kalman Filter propagates a deterministic set of sigma points through the exact nonlinear functions and reconstructs a Gaussian from them, needing no derivatives. Run both on the nonlinear simulator, compare their tracking error against the Step 1 baseline, and time them, because the UKF's accuracy gain usually costs a little more compute.
from filterpy.kalman import UnscentedKalmanFilter, MerweScaledSigmaPoints
pts = MerweScaledSigmaPoints(n=dim_x, alpha=1e-3, beta=2.0, kappa=0.0)
ukf = UnscentedKalmanFilter(dim_x=dim_x, dim_z=1, dt=1.0,
fx=f_nonlinear, hx=h_nonlinear, points=pts)
ukf.x, ukf.P, ukf.Q, ukf.R = x0, P0, Q, R
ukf_est = []
for zk in z:
ukf.predict(); ukf.update(zk) # sigma points through the true nonlinearity
ukf_est.append(ukf.x.copy())
Step 3: Run a particle filter on the non-Gaussian posterior
Drop the Gaussian assumption entirely, following Section 7.4. A particle filter represents the posterior with a weighted cloud of samples: it propagates each particle through the dynamics, reweights every particle by how well it explains the new measurement, and resamples when the effective sample size drops too low so that probable particles multiply and improbable ones die out. Run it on the same nonlinear problem, watch the effective sample size to confirm resampling is doing its job, and compare its tracking error to the Gaussian filters, especially where the posterior is skewed.
def particle_filter(z, f, h, Q, R, N=2000):
x = sample_prior(N) # initial particle cloud
est = []
for zk in z:
x = f(x) + np.random.normal(0, Q**0.5, x.shape) # propagate with process noise
w = gaussian_likelihood(zk, h(x), R) # reweight by the measurement
w /= w.sum()
if 1.0 / np.sum(w**2) < N / 2: # effective sample size below half
x = x[np.random.choice(N, N, p=w)] # resample to fight degeneracy
w = np.full(N, 1.0 / N)
est.append(np.sum(w * x)) # posterior-mean estimate
return np.array(est)
Step 4: Fit a hidden Markov model to a regime-switching variant
Switch from a continuous to a discrete hidden state, following Section 7.5. Generate a regime-switching version of the signal whose statistics jump between a small number of hidden regimes, then fit a Gaussian hidden Markov model that learns the transition probabilities and per-regime emission distributions with Baum-Welch and decodes the most likely hidden-state path with Viterbi. Confirm the recovered states line up with the true regimes; this is the same predict-update recurrence as the Kalman filter, run over discrete states.
from hmmlearn.hmm import GaussianHMM
hmm = GaussianHMM(n_components=2, covariance_type="full", n_iter=100)
hmm.fit(returns.reshape(-1, 1)) # Baum-Welch learns transitions and emissions
states = hmm.predict(returns.reshape(-1, 1)) # Viterbi most-likely hidden-state path
Step 5: Write the predict-update step as a tiny RNN cell
Close the loop by writing the filter as a recurrence, following Section 7.6. Strip the Kalman update to its essential shape, a state carried forward and corrected by the gap between prediction and observation, and write it as a minimal recurrent cell that takes the previous state and the current observation and returns the next state and a readout. Compare its structure to the Kalman equations of Step 1: the filter has known matrices and an analytic gain, the cell would learn them, but the carry-update-read shape is identical. This is the equivalence the chapter has been building toward.
def filter_cell(state, obs, A, B, K, C):
pred = A @ state # predict: roll the state forward
innov = obs - C @ pred # innovation: observation minus prediction
new_state = pred + K @ innov # update: correct by the gain-weighted gap
readout = C @ new_state # read out a prediction
return new_state, readout # an RNN learns A, B, K, C; the filter is given them
Expected Output
The study produces a ranked comparison rather than a single number. The linear baseline of Step 1 tracks the truth loosely and drifts wherever the nonlinearity bites, posting the highest tracking error. The Extended and Unscented filters of Step 2 both cut that error substantially, with the UKF typically edging out the EKF on the sharper nonlinearities at a modest runtime cost, the accuracy-versus-compute trade the section predicts. The particle filter of Step 3 matches or beats the Gaussian filters where the posterior turns skewed or multimodal, and its effective-sample-size trace shows resampling firing whenever the weights concentrate, at the price of being the slowest of the four. The hidden Markov model of Step 4 recovers the regime sequence with high agreement against the true labels, demonstrating that the same recurrence handles discrete hidden states. The recurrent cell of Step 5 reproduces the linear filter's estimate exactly when handed the Kalman matrices, making the punchline concrete: the four filters are four ways to specify one recurrence, and the recurrent network of Part III is the fifth, the one that learns the recurrence instead of being told it.
The lab leans on two libraries so the four-way comparison fits in a page of code. filterpy supplies KalmanFilter, ExtendedKalmanFilter, and UnscentedKalmanFilter with sigma-point helpers, the RTS smoother, and a particle-filter scaffold, all with a uniform predict-and-update interface, while hmmlearn supplies GaussianHMM and CategoricalHMM with Baum-Welch fitting and Viterbi decoding. For the linear-Gaussian state-space models of Section 7.1, statsmodels UnobservedComponents and SARIMAX expose the same Kalman machinery with maximum-likelihood parameter estimation built in. Write the Kalman filter, the particle cloud, and the recurrent cell from scratch once so you know what each library computes internally, then reach for filterpy, hmmlearn, and statsmodels whenever you track a hidden state for real.
Stretch Goals
- Run the RTS smoother after the Step 1 filter and confirm that smoothing past states in light of later observations beats the forward-only estimate, the smoothing task of Section 7.2.
- Push the particle count of Step 3 up and down and plot tracking error against runtime to map the Monte Carlo convergence the section describes, the effective-sample-size trade of Section 7.4.
- Replace the Gaussian HMM of Step 4 with a Markov-switching autoregression so each regime carries its own dynamics, and compare its regime recovery, the switching-regime extension of Section 7.5.
What's Next?
This chapter learned to infer a hidden state that evolves smoothly and to detect which of a few discrete regimes a series occupies. Both rely on a model of normal behavior, and both quietly raise a sharper question: what happens when the data stops behaving normally, when a measurement is an outlier the filter should distrust, or when the regime does not just switch but breaks, the underlying process changing for good? Chapter 8: Temporal Anomaly and Change-Point Detection takes up exactly that question. It uses the filtering machinery built here, a large innovation in a Kalman filter is itself an anomaly signal, and a shift in HMM regime occupancy is a change-point, to flag the points where a series departs from its model and the moments where the model itself must change. The hidden state stays central; the chapter simply asks when the state, or the dynamics that drive it, has broken.
Bibliography & Further Reading
Foundational Texts
Kalman, R. E. "A New Approach to Linear Filtering and Prediction Problems." Journal of Basic Engineering (ASME), 82(1), 1960. doi.org/10.1115/1.3662552
The paper that introduced the recursive linear-Gaussian filter, the predict-and-update recursion and minimum-variance optimality at the heart of Section 7.2 and the whole chapter.
Rabiner, L. R. "A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition." Proceedings of the IEEE, 77(2), 1989. ieeexplore.ieee.org
The canonical HMM tutorial covering the forward-backward algorithm, Viterbi decoding, and Baum-Welch learning, the reference treatment behind Section 7.5.
Key Books
Särkkä, S. "Bayesian Filtering and Smoothing." Cambridge University Press, 2013 (2nd edition 2023; free PDF). users.aalto.fi/~ssarkka
The modern unified treatment of Kalman, Extended, Unscented, and particle filters and smoothers under one Bayesian framework, the single best companion to Sections 7.2 through 7.4.
Durbin, J., Koopman, S. J. "Time Series Analysis by State Space Methods," 2nd edition. Oxford University Press, 2012. global.oup.com
The definitive statistical treatment of the linear-Gaussian state-space model, showing how ARIMA, ETS, and unobserved-components models all become one state-space form, the spine of Section 7.1.
Bishop, C. M. "Pattern Recognition and Machine Learning." Springer, 2006 (free PDF). microsoft.com/research
Chapter 13 develops linear dynamical systems (the Kalman filter) and hidden Markov models as two faces of one latent-sequence model, the machine-learning view that frames Sections 7.1, 7.2, and 7.5.
Papers
Julier, S. J., Uhlmann, J. K. "Unscented Filtering and Nonlinear Estimation." Proceedings of the IEEE, 92(3), 2004. ieeexplore.ieee.org
The definitive account of the unscented transform and the Unscented Kalman Filter, the sigma-point method that avoids the EKF's Jacobian, developed in Section 7.3.
Arulampalam, M. S., Maskell, S., Gordon, N., Clapp, T. "A Tutorial on Particle Filters for Online Nonlinear/Non-Gaussian Bayesian Tracking." IEEE Transactions on Signal Processing, 50(2), 2002. ieeexplore.ieee.org
The standard particle-filter tutorial covering importance sampling, the bootstrap filter, degeneracy, and resampling, the exact scope of Section 7.4 and the lab's Step 3.
Tools & Libraries
Labbe, R. "Kalman and Bayesian Filters in Python" and the filterpy library. github.com/rlabbe
A free, runnable, notebook-length book and its companion library implementing the Kalman, Extended, Unscented, and particle filters; the engine and the gentlest reference for the entire lab.
hmmlearn: hidden Markov models with scikit-learn-style API in Python. hmmlearn.readthedocs.io
Gaussian and categorical HMMs with Baum-Welch fitting and Viterbi decoding, the library behind Section 7.5 and the lab's regime-recovery step.
statsmodels: state-space models (UnobservedComponents, SARIMAX, DynamicFactor) in Python. statsmodels.org
The Kalman-filter-backed state-space framework that fits unobserved-components, ARIMA, and factor models by maximum likelihood, the production route for Sections 7.1 and 7.2.
particles: sequential Monte Carlo and particle filtering in Python (Nicolas Chopin). github.com/nchopin/particles
A research-grade library for particle filters, SMC samplers, and resampling schemes, the production-strength counterpart to the from-scratch filter of Section 7.4.
Datasets & Benchmarks
Johnson, A. E. W., et al. "MIMIC-III, a freely accessible critical care database." Scientific Data 3, 2016 (PhysioNet). physionet.org
A large database of irregularly sampled intensive-care vital signs, the realistic high-noise tracking benchmark for which the filters of this chapter and the lab's bedside-monitor framing are built.