"For thirty-four chapters they taught me one trick at a time, and I learned each in its own sealed room: forecasting in one, filtering in another, uncertainty down the hall, decisions somewhere upstairs. I thought the rooms were the point. Then on a Monday they put me in a hospital and a trading desk and a turbine and a power grid and a robot arm and a physics lab, all at once, and told me the walls had never been real. The clinician did not want my autoregression and my conformal interval and my policy as three separate gifts; she wanted one answer she could act on before the next reading came in. That was the day I understood the difference between knowing the techniques and shipping the system: the world does not hand you a clean benchmark and a single loss, it hands you a leaky pipeline, a baseline to beat, an uncertainty you must quantify, and a decision someone is waiting on."
A Generalist Forecaster on Its First Day Out of the Sandbox
Chapter Overview
Part IX opens here, and with it the book turns from teaching temporal intelligence to deploying it where the stakes are real. The previous parts built a toolkit one capability at a time: classical forecasting and filtering, deep sequence models and foundation models, representation learning, uncertainty quantification, sequential decision making, reasoning, and the trustworthiness and operations practices that make a model fit to ship. Each was developed in relative isolation, on a clean benchmark, judged by a single metric. This chapter is the payoff, where that isolation ends. It takes the book's running datasets and the full inventory of techniques built around them, then weaves them into end-to-end systems across eight high-consequence domains.
The unit of this chapter is not a method but a domain story. Each of the eight sections takes a real problem and walks it from raw, messy data to a decision someone acts on, and along the way it pulls whatever the problem actually needs from across the book rather than whatever a single chapter happened to teach. A finance pipeline blends an ARIMA or GARCH baseline with a deep forecaster, wraps the prediction in a calibrated interval, and feeds it to an execution policy. A clinical monitor filters an irregular multivariate stream, predicts deterioration with quantified uncertainty, and decides when to raise an alarm. A maintenance system detects change points in vibration telemetry, estimates remaining useful life, and schedules an intervention under a cost model. A demand-planning system turns calibrated quantiles into inventory decisions, while a security system ranks adversarial event streams into an analyst queue. The classical idea, the learned model, the uncertainty estimate, and the decision rule stop being separate chapters and become one system.
Eight domains structure the chapter: finance, healthcare, manufacturing and predictive maintenance, energy and climate, autonomous systems and robotics, scientific discovery, supply chain and demand planning, and cybersecurity and fraud. They are chosen to span the full range of temporal AI in practice: regression and classification, point and probabilistic forecasts, control and planning, structured physical priors and black-box learning, fast tick-level loops and slow seasonal cycles, benign demand shocks and adversarial behavior. The domains differ enormously in their data, their physics, and their consequences, yet the same skeleton runs through every one of them, which is precisely the point. Learning the skeleton, rather than memorizing eight recipes, is what lets you walk into a ninth domain the book never covered and build the system anyway.
Four disciplines recur in every section, and they are the real curriculum here. First, no leakage: in a temporal system the most expensive bugs are the ones that let future information seep into the past, and every pipeline in this chapter is built to respect the arrow of time from the first split onward. Second, beat the baseline: a deep model that cannot outperform a seasonal naive forecast or a simple ARIMA has earned nothing, and every domain story names the baseline it must beat. Third, quantify uncertainty: a point prediction with no honest interval is not deployable in any of these settings, and calibrated uncertainty is treated as a requirement rather than an extra. Fourth, decision-awareness: the metric that matters is the one tied to the action the system triggers, not the loss that was convenient to optimize, so each domain closes the loop from prediction to decision to outcome.
Prerequisites
This chapter is the one place in the book that draws on the whole of it, so rather than a short prerequisite list it assumes a working command of Parts II through VIII. The finance and energy stories rest on the classical forecasting of Chapter 5: Univariate Forecasting Models and the deep architectures that built on it, the clinical and maintenance stories on the filtering and anomaly detection of Part II and the deep sequence models of Part III. Every domain wraps its predictions in the calibrated intervals of Chapter 19: Probabilistic Forecasting and Uncertainty Quantification, because none of these systems ships a bare point estimate, and the finance-execution, maintenance-scheduling, and robotics stories close the loop with the sequential decision making of Chapter 22: Markov Decision Processes and the reinforcement learning and control that follow it. The trustworthiness and deployment practices of Part VIII run underneath all of it. Readers wanting to refresh any specific technique a domain reaches for, a model, an uncertainty method, a decision algorithm, will find the right chapter through the Table of Contents.
If you keep one idea from this chapter, keep this: shipping temporal AI in the real world is not about picking the single best technique but about assembling the whole toolkit into one leakage-safe, baseline-beating, uncertainty-aware, decision-driven pipeline, and the same skeleton, data preparation to model to calibrated uncertainty to decision to evaluation, runs through finance, healthcare, manufacturing, energy, robotics, science, supply chain, and security alike. Each domain reaches into a different corner of the book, a classical baseline here, a deep forecaster there, a probabilistic wrapper, a control policy, but the four disciplines never change: respect the arrow of time, beat the honest baseline, quantify what you do not know, and judge the system by the decision it triggers rather than the loss it was convenient to minimize.
Chapter Roadmap
- 35.1 Finance An end-to-end financial pipeline on the running returns series: forecasting price and return, modeling volatility and risk with GARCH and deep alternatives, wrapping predictions in calibrated intervals, and feeding them to an execution and risk-management policy, with leakage and overfitting treated as the central dangers.
- 35.2 Healthcare Clinical time series done right on the running irregular vitals stream: filtering and imputing an irregularly sampled multivariate signal, real-time monitoring and early-warning prediction with quantified uncertainty, prognosis and time-to-event modeling, and deciding when an alarm is worth raising.
- 35.3 Manufacturing and Predictive Maintenance From sensor telemetry to a maintenance schedule on the running industrial stream: change-point and anomaly detection on vibration and wear signals, remaining-useful-life estimation with uncertainty, and turning a degradation forecast into a cost-aware intervention decision before a machine fails.
- 35.4 Energy and Climate Forecasting under strong seasonality and physical structure: probabilistic load and renewable-generation forecasting, the GEFCom-style competition setting, weather and climate models from numerical prediction to learned emulators, and using calibrated forecasts to schedule and balance a grid.
- 35.5 Autonomous Systems and Robotics Temporal models that perceive, predict, and act: trajectory and motion forecasting, world models and planning over learned dynamics, imitation and reinforcement learning for control, and vision-language-action policies that turn a stream of observations into a stream of decisions.
- 35.6 Scientific Discovery Temporal AI as an instrument of science: discovering governing equations from data with sparse regression, physics-informed networks that fuse differential equations with learning, and learned simulators that accelerate forecasting of physical systems, all under uncertainty that scientists can interrogate.
- 35.7 Supply Chain, Retail Demand, and Inventory Decisions Demand forecasting as a decision system: hierarchy, intermittency, promotions, probabilistic forecasts, reconciliation, safety stock, service levels, disruptions, and foundation models for demand planning.
- 35.8 Cybersecurity, Fraud, and Temporal Security Operations Security operations as temporal decision systems over event streams, delayed labels, temporal graphs, anomaly detection, analyst queues, adversarial drift, and foundation models for anomaly detection.
Once you have worked through the eight domain stories, the Hands-On Lab below asks you to pick one of them and ship a complete pipeline of your own. Each lab step maps to the skeleton every section shares, so the lab doubles as a synthesis: by the time you finish you will have prepared data without leakage, beaten a named baseline, calibrated an uncertainty estimate, driven a decision, and evaluated the whole loop, reusing components drawn from across the book exactly as the domain section you chose did.
Hands-On Lab: Pick a Domain, Ship a Pipeline
Objective
Choose one of the eight domains from this chapter and build a complete, end-to-end, leakage-safe temporal pipeline for it, reusing components from across the whole book rather than writing anything new from scratch. The goal is not a clever model but a defensible system: data preparation that respects the arrow of time, a model that beats a named baseline, a calibrated uncertainty estimate around every prediction, a decision rule that turns predictions into actions, and an evaluation that judges the decision rather than the loss. If you pick finance, you might forecast returns and volatility following Section 35.1 and feed a sized position to an execution rule. If you pick healthcare, you might monitor the clinical stream of Section 35.2 and trigger an early-warning alarm. If you pick maintenance, you might detect degradation in the sensor stream of Section 35.3 and schedule an intervention; if energy, forecast load with calibrated intervals following Section 35.4; if robotics, predict a trajectory and plan over it following Section 35.5; if science, recover dynamics or build a learned emulator following Section 35.6; if supply chain, turn a demand distribution into an inventory decision following Section 35.7; if security, rank time-critical alerts under label delay following Section 35.8. Whichever you choose, the single thread is that a deployable system is a chain, and it is only as strong as its weakest link.
What You'll Practice
- Building a leakage-safe temporal split and feature pipeline for a real domain, the discipline shared by all of Sections 35.1 through 35.8.
- Selecting and beating an honest baseline appropriate to the domain you chose, never shipping a model that has not earned its complexity.
- Wrapping every prediction in a calibrated uncertainty estimate before it reaches a decision, the requirement common to all eight sections.
- Turning predictions and their uncertainty into an explicit decision rule tied to a domain cost or utility, the decision-awareness of Section 35.5 and the rest of the chapter.
- Evaluating the whole loop on a decision-relevant metric and reporting where the chain is weakest.
Setup
You need a Python environment with the libraries the book has used throughout: a forecasting stack such as statsmodels, statsforecast, or darts, torch for any deep component, a conformal or quantile tool for the calibrated intervals of Section 35.1 through Section 35.4, and, if your domain involves control, gymnasium with stable-baselines3 for the decision policy of Section 35.5. Use one of the book's running datasets aligned to your chosen domain, the returns series for finance, the clinical vitals for healthcare, the sensor telemetry for maintenance, or a public domain dataset from the bibliography below. The one discipline that governs the whole lab is to build the pipeline as an ordered chain and to refuse to advance a stage until the previous one is honest: do not tune a model before the split is leakage-free, do not calibrate before the model beats its baseline, and do not evaluate the decision before the uncertainty is calibrated. The code below sketches the leakage-safe time-ordered split that every domain pipeline begins with.
import numpy as np
def temporal_split(series, train_frac=0.7, val_frac=0.15):
"""Split a time series in temporal order, never shuffled.
Train precedes validation precedes test, so no future leaks into the past."""
n = len(series)
i_train = int(n * train_frac) # train ends here
i_val = int(n * (train_frac + val_frac)) # validation ends here
train = series[:i_train] # earliest segment
val = series[i_train:i_val] # tune and calibrate here
test = series[i_val:] # touched once, at the very end
return train, val, test
Steps
Step 1: Prepare the data without leakage
Load your chosen domain's data, build features that use only past information at each timestep, and split it in strict temporal order with the helper above. Fit every scaler, imputer, and encoder on the training segment alone and apply it forward, the no-leakage discipline that opens every section from 35.1 to 35.8. Write down exactly what each feature would have known at prediction time.
Step 2: Establish and beat a baseline
Choose the honest baseline for your domain, a seasonal naive or ARIMA forecast for finance or energy, a last-observation or simple-threshold rule for monitoring or maintenance, and measure it first. Then train your model and confirm it beats the baseline on the validation segment before you spend any more effort on it, the beat-the-baseline rule the whole chapter insists on.
Step 3: Calibrate the uncertainty
Wrap your model's predictions in a calibrated interval or predictive distribution using a conformal or quantile method on the validation segment, following the uncertainty discipline of Section 35.1 through Section 35.4. Check empirical coverage: a stated ninety percent interval should contain the truth about ninety percent of the time, and if it does not, fix the calibration before going further.
Step 4: Drive a decision
Define the decision your system exists to make, a position size, an alarm, a maintenance order, a dispatch schedule, a control action, and write the rule that maps a calibrated prediction to that action under an explicit cost or utility, the decision-awareness of Section 35.5. Let the uncertainty change the action: a wide interval should make the system act more cautiously, not ignore its own doubt.
Step 5: Evaluate the whole loop
Run the full chain once on the held-out test segment and score it on a decision-relevant metric, realized profit and drawdown, alarm precision and lead time, downtime avoided against intervention cost, dispatch cost, control return, not on the training loss. Report which link in the chain is weakest and what you would strengthen next, the honest verdict every domain section in this chapter ultimately demands.
Expected Output
The lab produces one complete domain pipeline you can defend stage by stage. Step 1 yields a leakage-audited dataset and a written account of what each feature knew at prediction time, the single most common place these systems silently cheat. Step 2 yields a named baseline and a model that demonstrably beats it on validation, with the comparison made on the same split rather than two convenient configs. Step 3 yields calibrated predictions whose empirical coverage you have checked, so the uncertainty is a measured property rather than a hopeful label. Step 4 yields an explicit decision rule that consumes the uncertainty rather than discarding it, closing the gap between a forecast and an action. Step 5 yields a decision-relevant evaluation on data the pipeline saw exactly once, together with an honest statement of the weakest link. The reader finishes having shipped, not just studied, a temporal system, and, more importantly, having internalized that the skeleton transfers: the same five steps would carry you through any of the other seven domains, and through a ninth this book never named.
Building the chain by hand once teaches you where leakage and miscalibration hide, but in practice most of the pipeline is glue that mature libraries already provide. For forecasting with proper backtesting and time-ordered cross-validation, darts and Nixtla's statsforecast and neuralforecast supply the models, the rolling-origin evaluation, and the baselines of Section 35.1 and Section 35.4 in a few lines, and they make the leakage-safe split of Step 1 the default rather than something you must remember. For calibrated intervals, conformal wrappers around any of these collapse Step 3 into a single call. For the decision and control side of Section 35.5, stable-baselines3 and d3rlpy turn a defined environment and reward into a trained policy without your reimplementing the algorithm. The discipline the chapter teaches still governs the libraries: a backtest is only honest if the split it hides was leakage-free, a conformal interval is only valid if the calibration data was truly held out, and a policy is only useful if the reward encodes the decision you actually care about. The library writes the glue; it does not decide whether you measured the right thing, and that decision is the whole job.
Stretch Goals
- Build the same pipeline for a second domain from the chapter and reuse as much code as possible, proving to yourself that the skeleton of Section 35.1 through Section 35.8 transfers and isolating the few parts that are genuinely domain-specific.
- Stress your pipeline against distribution drift by evaluating it on a later, regime-shifted segment than the one it was calibrated on, exposing whether the calibrated coverage of Step 3 survives the kind of non-stationarity every real deployment eventually meets.
- Swap your bespoke model for a temporal foundation model in zero-shot or fine-tuned form and ask whether it beats your domain-specific baseline, testing the claim that pretrained temporal models are starting to change the build-from-scratch calculus across all eight domains.
What's Next?
This chapter put the whole toolkit to work, taking the book's running datasets and the full inventory of techniques and assembling them into deployable systems across eight domains, each held to the same disciplines of no leakage, a beaten baseline, calibrated uncertainty, and a decision worth making. Having seen how far the present tools reach, the natural question is where they are heading. Chapter 36: Toward General Temporal Intelligence takes the final step, looking past the domain-specific pipelines of this chapter to the prospect of temporal models that generalize across tasks and domains the way language models generalize across text: foundation models for time, the open problems that still separate narrow forecasters from general temporal reasoners, and the research frontier that will define the next decade of the field. The skeleton you shipped in this chapter is the thing those general systems aim to internalize, and the disciplines you practiced are the ones they will still have to honor. The full path through Part IX and the rest of the book is laid out in the Table of Contents.
Bibliography & Further Reading
Finance
Lopez de Prado, M. "Advances in Financial Machine Learning." Wiley, 2018. wiley.com
The standard practitioner text on machine learning for finance, with a sustained focus on the leakage, labeling, and backtest-overfitting pitfalls that the finance pipeline of Section 35.1 is built to avoid.
Healthcare
Johnson, A. E. W., Pollard, T. J., Shen, L., Lehman, L.-W. H., Feng, M., Ghassemi, M., Moody, B., Szolovits, P., Celi, L. A., Mark, R. G. "MIMIC-III, a Freely Accessible Critical Care Database." Scientific Data, 2016. physionet.org/content/mimiciii
The widely used public ICU database of irregularly sampled multivariate clinical time series, the kind of stream the monitoring and prognosis pipeline of Section 35.2 is designed to filter, predict over, and act on.
Goldberger, A. L., et al. "PhysioNet: Components of a New Research Resource for Complex Physiologic Signals." Circulation, 2000. physionet.org
The open repository of physiologic signal databases and challenges, the source of the clinical and waveform benchmarks behind the healthcare monitoring tasks of Section 35.2.
Manufacturing and Predictive Maintenance
Saxena, A., Goebel, K., Simon, D., Eklund, N. "Damage Propagation Modeling for Aircraft Engine Run-to-Failure Simulation (C-MAPSS Turbofan Dataset)." PHM, 2008. NASA Prognostics Data Repository. nasa.gov/pcoe-data-set-repository
The benchmark run-to-failure turbofan dataset behind much of the remaining-useful-life literature, the canonical testbed for the degradation detection and maintenance-scheduling pipeline of Section 35.3.
Energy and Climate
Hong, T., Pinson, P., Fan, S., Zareipour, H., Troccoli, A., Hyndman, R. J. "Probabilistic Energy Forecasting: Global Energy Forecasting Competition 2014 and Beyond." International Journal of Forecasting, 2016. doi.org/10.1016/j.ijforecast.2016.02.001
The reference for the GEFCom2014 probabilistic load, price, wind, and solar forecasting competition, the competition setting and evaluation that frame the calibrated energy forecasting of Section 35.4.
Lam, R., et al. "Learning Skillful Medium-Range Global Weather Forecasting (GraphCast)." Science, 2023. arXiv:2212.12794. arxiv.org/abs/2212.12794
The graph-neural-network weather model that matches or beats operational numerical prediction at a fraction of the cost, the learned-emulator paradigm at the climate end of Section 35.4.
Bi, K., Xie, L., Zhang, H., Chen, X., Gu, X., Tian, Q. "Accurate Medium-Range Global Weather Forecasting with 3D Neural Networks (Pangu-Weather)." Nature, 2023. doi.org/10.1038/s41586-023-06185-3
A three-dimensional transformer weather model delivering fast, accurate medium-range forecasts, further evidence for the learned-simulator approach to climate forecasting in Section 35.4.
Autonomous Systems and Robotics
Brohan, A., et al. "RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control." arXiv, 2023. arXiv:2307.15818. arxiv.org/abs/2307.15818
The vision-language-action model that turns a stream of observations and instructions into robot actions, the perceive-predict-act policy paradigm at the center of Section 35.5.
Scientific Discovery
Brunton, S. L., Proctor, J. L., Kutz, J. N. "Discovering Governing Equations from Data by Sparse Identification of Nonlinear Dynamical Systems (SINDy)." PNAS, 2016. doi.org/10.1073/pnas.1517384113
The sparse-regression method that recovers interpretable governing equations directly from time-series measurements, the equation-discovery tool that opens Section 35.6.
Raissi, M., Perdikaris, P., Karniadakis, G. E. "Physics-Informed Neural Networks." Journal of Computational Physics, 2019. arXiv:1711.10561. arxiv.org/abs/1711.10561
The framework that embeds differential-equation constraints into a neural network's loss, fusing physical priors with data-driven learning, the physics-informed approach of Section 35.6.
Supply Chain and Demand Planning
"Foundation Models for Demand Forecasting via Dual-Strategy Ensembling." arXiv, 2025. arxiv.org/abs/2507.22053
A recent demand-forecasting foundation-model study that motivates the hierarchy-aware, architecture-aware, and decision-linked retail planning discussion in Section 35.7.
Makridakis, S., Spiliotis, E., Assimakopoulos, V. "The M5 Accuracy Competition: Results, Findings and Conclusions." International Journal of Forecasting, 2022. doi.org/10.1016/j.ijforecast.2021.11.013
The large-scale hierarchical retail demand benchmark behind the reconciliation, WRMSSE, and inventory-decision framing of Section 35.7.
Cybersecurity and Fraud
"Foundation Models for Anomaly Detection: A Survey." arXiv, 2025. arxiv.org/abs/2502.06911
A current survey of anomaly foundation models as encoders, detectors, and interpreters, the model-family frontier used in Section 35.8.
"LLM-Based Event Log Analysis: A Survey." arXiv, 2025. arxiv.org/abs/2502.00677
A recent survey of log parsing, sequence modeling, and LLM-assisted event-log analysis, supporting the SOC and log-anomaly parts of Section 35.8.
Cross-Domain Benchmarks and Data
Godahewa, R., Bergmeir, C., Webb, G. I., Hyndman, R. J., Montero-Manso, P. "Monash Time Series Forecasting Archive." NeurIPS Datasets and Benchmarks, 2021. forecastingdata.org
A large open archive of forecasting datasets spanning energy, finance, traffic, health, and more with standard baselines, the cross-domain benchmark collection that supports the comparative spirit of every section in this chapter.