"For ten thousand timesteps I behaved exactly as the model expected, and nobody looked at me twice. Then for one timestep I spiked, three sigma above where the forecast said I should be, and suddenly I was the most interesting number in the whole dashboard. The thing is, I was not wrong about anything important; I was just surprising, and surprise is the only currency a monitor trades in. Score me by how badly your model of normal failed to predict me, threshold that score, and you will catch me. Wait a little longer and you will face the harder question: was I a one-off, or was I the first sign that the rules you have been scoring me against quietly stopped being true?"
A Residual That Spiked at Exactly the Wrong Moment
Chapter Overview
A monitored series asks two questions, and they are not the same question. The first is local and momentary: did this single observation, or this short stretch, fail to belong, given everything the series has done so far? That is anomaly detection, the search for brief deviations, a sensor reading that jumps, a transaction that does not fit the account's habits, a vital sign that lurches between two otherwise normal measurements. The second question is structural and lasting: did the process generating the data change, so that what counted as normal yesterday is the wrong yardstick today? That is change-point detection, the search for persistent regime shifts, a machine wearing into a new operating mode, a market moving to a higher-volatility regime, a service whose baseline latency steps up after a deployment. This chapter takes the sensor and IoT telemetry stream as its running dataset, following the Part II rotation, and uses it to make both questions concrete: an industrial signal that mostly behaves, occasionally spikes, and now and then changes its character for good.
The chapter follows the natural progression of the field. It opens with the taxonomy that time forces on the problem, point, contextual, and collective anomalies, and the reason an unremarkable value can still be an anomaly once its temporal context is taken into account. From there it builds the classical scoring machinery: statistical thresholds from the robust z-score and the Grubbs and extreme-studentized-deviate tests, the streaming control charts (Shewhart, CUSUM, EWMA) that industrial monitoring grew up on, and the distance and density methods (k-nearest-neighbor distance, the Local Outlier Factor, Isolation Forest, and the matrix profile for subsequence discords) that score how isolated a point or a window is. Then it turns from anomalies to change points proper, casting segmentation as a cost-plus-penalty optimization solved exactly by PELT and approximately by binary segmentation in the offline setting, and as a run-length posterior tracked by Bayesian Online Change-Point Detection in the streaming setting, with detection delay traded against the false-alarm rate.
The modern framing then unifies almost everything that came before. Anomaly detection becomes the scoring of large prediction error or large reconstruction error: feed the series to a forecaster, an ARIMA model or the Kalman filter of Chapter 7, and a point your model of normal did not expect shows up as a large residual, while a regime change shows up as a run of residuals that stays large. This is where the temporal thread runs forward. The reconstruction-error idea, encode the recent window into a compact representation and flag whatever the representation cannot rebuild, is exactly the principle behind the autoencoders and variational autoencoders of Part IV; the chapter's classical PCA reconstruction is their linear ancestor, and a large Kalman innovation is a forecasting residual by another name. Scoring surprise and thresholding it is the single idea that the deep detectors of Parts III and IV will inherit and learn rather than hand-build.
This chapter sits at a hinge. It closes Part II by collecting the classical view of anomalies and change points and showing that the methods of the preceding chapters, decomposition, forecasting, and filtering, are already half of an anomaly detector once you read their residuals as surprise scores. At the same time it bridges to the rest of the book: the reconstruction framing points to deep generative detectors, the streaming and delay concerns point to the online and drift-aware learning of Part V, and the monitoring applications point to the deployment and observability concerns of Part VIII. The final section grounds all of it in production, where the real adversary is not a missed anomaly but alert fatigue, and where precision, recall, and detection delay must be traded against one another under asymmetric costs.
Prerequisites
This chapter draws on three earlier ones. From Chapter 3: Statistical Foundations you need decomposition and stationarity, because the honest way to score an anomaly is to deseasonalize and detrend first and then ask whether the residual is surprising, and because a change point is most cleanly defined as a break in an otherwise stationary segment. From Chapter 5: Univariate Forecasting Models you need ARIMA and exponential smoothing, since the chapter's modern framing scores anomalies from forecasting residuals and a forecaster you can already fit is the front half of a detector. From Chapter 7: State-Space Models and Filtering you need the Kalman innovation, the one-step prediction error the filter produces for free, because a large innovation is an anomaly signal and a sustained shift in innovations is a change point, the cleanest link between filtering and detection. A little comfort with the precision-and-recall vocabulary of binary classification helps, since detectors are evaluated as imbalanced classifiers. Readers wanting a refresher on any of these will find the relevant chapters and the probability and statistics appendices indexed in the Table of Contents.
If you keep one idea from this chapter, keep this: an anomaly is a point your model of normal did not expect; a change point is the moment your model of normal stopped being true; and both reduce to scoring surprise and thresholding it. Build a model of normal, by a fitted threshold, a distance, a forecaster, or a reconstruction, score each new point or window by how badly that model failed to predict it, and raise an alert when the score crosses a threshold you set to balance false alarms against missed events. An anomaly is a single large score; a change point is a score that stays large because the model itself is now wrong. The deep detectors of Parts III and IV will not change this recipe; they will only learn a richer model of normal, and the reconstruction error at the center of the modern framing is the direct ancestor of the autoencoder.
Chapter Roadmap
- 8.1 Point, Contextual, and Collective Anomalies Why time turns anomaly detection into a context-dependent problem, the taxonomy of point, contextual, and collective anomalies, the supervised, unsupervised, and semi-supervised framings under extreme class imbalance, the difference between scoring and labeling, and how to evaluate a detector honestly with precision, recall, and point-adjusted metrics.
- 8.2 Statistical and Distance-Based Approaches The robust z-score and median-absolute-deviation thresholds, the Grubbs and extreme-studentized-deviate tests, Shewhart, CUSUM, and EWMA control charts for streaming detection, k-nearest-neighbor distance and Local Outlier Factor density methods, Isolation Forest, and the matrix profile for subsequence discords, all applied after a leakage-safe deseasonalize-then-detect step.
- 8.3 Change-Point Detection (offline and online) Distinguishing brief anomalies from persistent regime shifts, the cost-plus-penalty segmentation framework, exact dynamic programming with PELT and approximate binary segmentation offline, the likelihood-ratio and CUSUM view of a single change, Bayesian Online Change-Point Detection with its run-length posterior, and the detection-delay versus false-alarm trade-off.
- 8.4 Forecasting-Residual and Reconstruction Methods The unifying modern framing of detection as large prediction error or large reconstruction error: forecasting-residual scores from ARIMA and the Kalman innovation, reconstruction-based detection with PCA and the bridge to autoencoders, dynamic and extreme-value-theory thresholds, and the place of the classical methods inside the modern deep landscape.
- 8.5 Industrial and Monitoring Applications Putting the chapter's detectors to work in production: predictive maintenance and condition monitoring on sensor streams, IT observability and security monitoring at scale, healthcare and finance early-warning under asymmetric costs, and the operational reality of alert fatigue and the precision, recall, and delay trade-offs that govern a real monitor.
Once you have worked through the five sections, the Hands-On Lab below chains them into a single streaming monitor. Each lab step maps to a section, so the lab doubles as a review: by the time you finish you will have scored anomalies with the statistical, distance, and forecasting-residual methods of Section 8.1, Section 8.2, and Section 8.4, detected a regime change with the offline and online change-point algorithms of Section 8.3, and confronted the threshold, cooldown, and evaluation realities of Section 8.5.
Hands-On Lab: Build a Streaming Monitor
Objective
Take one sensor stream and turn it into a working monitor that scores anomalies three different ways, catches a regime change, and is evaluated honestly against injected ground truth. You will start from an industrial telemetry series, a seasonal signal with a daily cycle, a slow drift, occasional injected spikes, and one persistent regime change partway through, with every injected event recorded so you can measure detection quality directly. First you will deseasonalize and detrend the stream so that the residual is what you actually score, the leakage-safe preprocessing that makes every later step honest. Then you will score anomalies three ways and compare them: a robust z-score combined with a CUSUM control chart for the cheap streaming baseline, a matrix-profile discord score for subsequence anomalies that no single-point method can see, and a forecasting-residual score from a one-step forecaster, the modern framing in which an anomaly is simply a large prediction error. Next you will detect the persistent regime change two ways, offline with ruptures and online with Bayesian Online Change-Point Detection, and compare the detection delay each incurs. Then you will combine the scores under a dynamic threshold with a cooldown so that one event does not fire a hundred alerts, the single most important defense against alert fatigue. Finally you will evaluate the whole monitor against the injected ground truth, reporting precision, recall, and detection delay, the three numbers that decide whether a monitor is trusted in production. The lab is the chapter in miniature: one stream, three anomaly scores, two change-point detectors, one threshold, and one honest evaluation.
What You'll Practice
- Deseasonalizing and detrending a sensor stream so the residual is the thing you score, following Section 8.1.
- Scoring anomalies with a robust z-score and a CUSUM control chart and with a matrix-profile discord, following Section 8.2.
- Scoring anomalies as forecasting residuals from a one-step forecaster, the modern prediction-error framing, following Section 8.4.
- Detecting a persistent regime change offline with ruptures and online with Bayesian Online Change-Point Detection, and comparing detection delay, following Section 8.3.
- Setting a dynamic threshold with a cooldown and evaluating precision, recall, and detection delay against injected ground truth, following Section 8.5.
Setup
You need stumpy for the matrix-profile discord score, ruptures for offline change-point detection, statsmodels for the seasonal decomposition and the one-step forecaster, and NumPy and SciPy for the robust z-score, the CUSUM chart, and the from-scratch Bayesian Online Change-Point Detection run-length recursion. A small simulator for the sensor stream and its injected spikes and regime change ships with the lab so it runs out of the box and so every alert can be scored against a known label. Every step respects the leakage discipline of Chapter 2: the monitor sees each observation once, in order, and the deseasonalize-then-detect step never fits on the future it is about to score.
pip install stumpy ruptures statsmodels numpy scipy matplotlibSteps
Step 1: Deseasonalize the stream and define the residual
Start by stripping out everything that is normal so that what remains is what you score, following Section 8.1. Decompose the sensor stream into its seasonal cycle, its slow trend, and a residual, fitting the seasonal and trend components only on past data so the step stays leakage-free, and keep the residual as the signal every later detector consumes. A value that looks high in the raw stream may be perfectly normal for its hour of the day; only the residual reveals the contextual anomalies the chapter opens with.
from statsmodels.tsa.seasonal import STL
import numpy as np
def residualize(stream, period):
stl = STL(stream, period=period, robust=True) # robust to the very anomalies we hunt
res = stl.fit()
residual = stream - res.seasonal - res.trend # what is left after normal is removed
return residual, res # score the residual, not the raw stream
Step 2: Score anomalies with a robust z-score and a CUSUM chart
Score the residual the cheap streaming way first, following Section 8.2. A robust z-score using the median and the median absolute deviation flags isolated point anomalies without letting a few outliers inflate the scale, while a CUSUM control chart accumulates small persistent deviations that no single-point test would catch, the streaming workhorse of industrial monitoring. Run both over the residual and keep their scores; they are the baseline the richer detectors must justify themselves against.
def robust_z(residual):
med = np.median(residual)
mad = np.median(np.abs(residual - med)) + 1e-9
return 0.6745 * (residual - med) / mad # MAD-based z-score, outlier-resistant scale
def cusum(residual, k):
s_pos = np.maximum.accumulate(np.zeros_like(residual)) # placeholder, filled in the loop
s_pos, s_neg, hi, lo = 0.0, 0.0, [], []
for r in residual:
s_pos = max(0.0, s_pos + r - k) # accumulate upward drift past slack k
s_neg = min(0.0, s_neg + r + k) # accumulate downward drift
hi.append(s_pos); lo.append(s_neg)
return np.array(hi), np.array(lo) # sustained deviation shows up as a rising sum
Step 3: Score subsequence anomalies with the matrix profile
Catch the anomalies that live in shape, not in a single value, following Section 8.2. The matrix profile slides a window across the series and records, for each position, the distance to its nearest neighbor elsewhere in the series; a window with no close match anywhere, a discord, is a subsequence anomaly that point methods miss entirely. Compute the matrix profile over the residual with stumpy and read the largest profile values as the discord scores.
import stumpy
def discord_score(residual, window):
mp = stumpy.stump(residual.astype(float), m=window) # nearest-neighbor distance per window
profile = mp[:, 0].astype(float) # the matrix profile itself
return profile # large values are discords, the rarest shapes
Step 4: Score anomalies as forecasting residuals
Score anomalies the modern way, as prediction error, following Section 8.4. Fit a one-step forecaster to the deseasonalized stream and, at each step, compare what it predicted to what actually arrived; a large forecasting residual is an anomaly by the chapter's unifying definition, a point the model of normal did not expect. This is the same residual a Kalman innovation gives for free, and it is the linear ancestor of the deep reconstruction detectors of Part IV.
from statsmodels.tsa.arima.model import ARIMA
def forecast_residual_score(stream, order=(2, 0, 1)):
scores = np.zeros(len(stream))
for t in range(50, len(stream)):
model = ARIMA(stream[:t], order=order).fit() # fit only on the past, never the future
pred = model.forecast(steps=1)[0] # one-step-ahead prediction of normal
scores[t] = stream[t] - pred # the residual is the anomaly score
return scores # large prediction error means surprise
Step 5: Detect the regime change offline and online
Move from brief anomalies to the persistent regime change, and detect it two ways, following Section 8.3. Offline, ruptures solves the cost-plus-penalty segmentation with PELT and hands you the change point after seeing the whole series, the accurate retrospective answer. Online, a Bayesian Online Change-Point Detection run-length recursion updates a posterior over how long the current regime has lasted and signals a change as soon as that posterior collapses, the answer you can act on in real time. Compare the detection delay each one incurs against the true change location.
import ruptures as rpt
def offline_changepoint(residual, penalty):
algo = rpt.Pelt(model="rbf").fit(residual) # exact cost-plus-penalty segmentation
return algo.predict(pen=penalty) # change-point indices, seen in hindsight
# online BOCPD signals as soon as the run-length posterior over the current regime collapses
Step 6: Threshold with a cooldown and evaluate
Turn scores into alerts and judge the result honestly, following Section 8.5. Combine the anomaly scores, set a dynamic threshold that adapts to the recent score distribution rather than a fixed line, and impose a cooldown so a single event cannot fire a burst of alerts, the practical fix for alert fatigue. Then score the firing alerts against the injected ground truth: report precision and recall on the point anomalies and detection delay on the regime change, the three numbers that decide whether the monitor earns its place in production.
def alerts_with_cooldown(score, threshold_fn, cooldown):
alerts, last = [], -cooldown
for t, s in enumerate(score):
thr = threshold_fn(score[:t]) # dynamic threshold from recent scores
if s > thr and (t - last) >= cooldown: # fire only if past the cooldown window
alerts.append(t); last = t # one event, one alert, not a hundred
return alerts # evaluate these against injected truth
Expected Output
The monitor produces a comparison, not a single verdict. The robust z-score and CUSUM baseline of Step 2 catches the obvious injected spikes cheaply but misses the subsequence anomalies whose individual values look normal, and it is prone to firing repeatedly on one event before the cooldown is added. The matrix-profile discord of Step 3 catches exactly those shape anomalies the z-score missed, at a higher compute cost. The forecasting-residual score of Step 4 catches both kinds whenever the forecaster's model of normal is good, and it degrades gracefully where the forecaster is uncertain, the behavior the modern framing predicts. On the regime change of Step 5, PELT recovers the change location precisely in hindsight while the Bayesian online recursion flags it a few timesteps later, the delay being the price of a real-time answer, and the gap between the two quantifies that price. After the dynamic threshold and cooldown of Step 6, precision rises sharply because the burst of duplicate alerts collapses to one per event, and the final report, precision and recall on the spikes and detection delay on the regime change, makes the central trade-off of the chapter concrete: tighten the threshold and you trade recall for precision, loosen it and you trade precision for recall, and the cooldown buys precision almost for free.
The lab leans on a small stack so the three-way scoring and two-way change-point detection fit in a page of code. stumpy computes the matrix profile and its discords with a uniform stump interface and scales to long streams; ruptures supplies Pelt, Binseg, and Window for offline change-point detection under a choice of cost models and penalties; statsmodels supplies the STL decomposition and the ARIMA one-step forecaster whose residual is the anomaly score. For the statistical and distance detectors of Section 8.2, PyOD wraps LOF, KNN, IForest, and dozens more behind one scikit-learn-style API, and for streaming work river offers online versions. Write the robust z-score, the CUSUM chart, and the Bayesian online recursion from scratch once so you know what each detector computes, then reach for stumpy, ruptures, PyOD, and river whenever you monitor a stream for real.
Stretch Goals
- Replace the ARIMA forecaster of Step 4 with the Kalman filter of Chapter 7 and score anomalies from its innovation, confirming that a large innovation is a forecasting residual by another name, the filtering-to-detection link of Section 8.4.
- Swap the matrix-profile window length of Step 3 up and down and watch which anomalies appear and disappear, mapping how subsequence detection depends on the assumed anomaly scale, the discord sensitivity of Section 8.2.
- Add a second injected regime change and a near-duplicate spike, then re-evaluate precision, recall, and delay to see how the dynamic threshold and cooldown of Step 6 hold up under a busier stream, the alert-fatigue stress test of Section 8.5.
What's Next?
This chapter closes Part II. Over six chapters the classical view of time series has been built from the ground up: decomposition and stationarity, the frequency domain, univariate and multivariate forecasting, state-space filtering, and now the detection of anomalies and change points, with the recurring discovery that a residual read as a surprise score turns any forecaster or filter into half of a detector. The classical methods are powerful, interpretable, and often enough, but every one of them hand-specifies its model of normal, and the world is full of series whose normal is too rich to write down by hand. Part III: Temporal Deep Learning takes up that challenge by learning the model of normal from data. It opens with Chapter 9: Neural Sequence Modeling Fundamentals, which lays the groundwork, sequences as tensors, the encoder-decoder view, teacher forcing, and the loss surfaces of sequence models, before the recurrent, convolutional, and attention architectures that follow. The reconstruction error you scored by hand in this chapter is about to be learned; the recurrence you met in Chapter 7 is about to become a network. Continue from the Table of Contents.
Bibliography & Further Reading
Foundational Texts
Chandola, V., Banerjee, A., Kumar, V. "Anomaly Detection: A Survey." ACM Computing Surveys, 41(3), 2009. doi.org/10.1145/1541880.1541882
The survey that fixed the field's vocabulary, point, contextual, and collective anomalies and the supervised, unsupervised, and semi-supervised framings, the taxonomy that organizes Section 8.1.
Montgomery, D. C. "Introduction to Statistical Quality Control," 8th edition. Wiley, 2019. wiley.com
The standard reference on Shewhart, CUSUM, and EWMA control charts, the streaming statistical machinery of Section 8.2 and the historical root of industrial monitoring.
Surveys
Blázquez-García, A., Conde, A., Mori, U., Lozano, J. A. "A Review on Outlier/Anomaly Detection in Time Series Data." ACM Computing Surveys, 54(3), 2021 (arXiv:2002.04236). arxiv.org/abs/2002.04236
A modern, time-series-specific taxonomy of outlier and anomaly detection methods, the up-to-date map across the statistical, distance, and prediction-based approaches of Sections 8.2 and 8.4.
Truong, C., Oudre, L., Vayatis, N. "Selective Review of Offline Change Point Detection Methods." Signal Processing, 167, 2020 (arXiv:1801.00718). arxiv.org/abs/1801.00718
The review behind the ruptures library, casting change-point detection as cost-plus-penalty segmentation and surveying PELT and binary segmentation, the spine of Section 8.3.
Papers
Adams, R. P., MacKay, D. J. C. "Bayesian Online Changepoint Detection." 2007 (arXiv:0710.3742). arxiv.org/abs/0710.3742
The run-length-posterior recursion at the heart of online change-point detection, the streaming algorithm of Section 8.3 and the lab's real-time detector.
Killick, R., Fearnhead, P., Eckley, I. A. "Optimal Detection of Changepoints With a Linear Computational Cost." Journal of the American Statistical Association, 107(500), 2012. doi.org/10.1080/01621459.2012.737745
The PELT algorithm that solves exact penalized segmentation in linear time, the offline change-point engine of Section 8.3 and the lab's retrospective detector.
Breunig, M. M., Kriegel, H.-P., Ng, R. T., Sander, J. "LOF: Identifying Density-Based Local Outliers." ACM SIGMOD, 2000. doi.org/10.1145/342009.335388
The Local Outlier Factor, scoring a point by the density of its neighborhood relative to its neighbors', the density method of Section 8.2.
Liu, F. T., Ting, K. M., Zhou, Z.-H. "Isolation Forest." IEEE ICDM, 2008. doi.org/10.1109/ICDM.2008.17
An anomaly detector that isolates rare points with random partitions instead of modeling density, fast and scalable, one of the distance-and-isolation methods of Section 8.2.
Yeh, C.-C. M., et al. "Matrix Profile I: All Pairs Similarity Joins for Time Series." IEEE ICDM, 2016. cs.ucr.edu/~eamonn/MatrixProfile
The matrix profile, a nearest-neighbor distance per subsequence whose largest values are discords, the subsequence-anomaly method of Section 8.2 and the lab's Step 3.
Wu, R., Keogh, E. J. "Current Time Series Anomaly Detection Benchmarks Are Flawed and Are Creating the Illusion of Progress." IEEE TKDE, 2021 (arXiv:2009.13807). arxiv.org/abs/2009.13807
A pointed critique of popular anomaly benchmarks, trivial cases, mislabeling, and run-to-failure bias, the evaluation caution that informs Sections 8.1 and 8.5.
Tools & Libraries
ruptures: change-point detection in Python (Truong, Oudre, Vayatis). centre-borelli.github.io/ruptures-docs
PELT, binary segmentation, and window methods under a uniform cost-and-penalty interface, the offline change-point engine of Section 8.3 and the lab's Step 5.
stumpy: scalable matrix-profile computation in Python. stumpy.readthedocs.io
A fast, parallel matrix-profile library for motifs and discords on long streams, the subsequence-anomaly tool behind Section 8.2 and the lab's discord score.
PyOD: a Python toolbox for outlier and anomaly detection. pyod.readthedocs.io
A unified scikit-learn-style API over LOF, k-NN, Isolation Forest, and dozens of other detectors, the production route for the statistical and distance methods of Section 8.2.
Datasets & Benchmarks
Lavin, A., Ahmad, S. "The Numenta Anomaly Benchmark (NAB)." 2015 onward. github.com/numenta/NAB
A labeled corpus of real streaming data with a scoring rule that rewards early detection, the streaming anomaly benchmark closest to the monitoring framing of Section 8.5.