Part Overview
Part III is the longest part of the book and its turning point: the moment temporal modeling becomes learned end to end. Seven chapters trace the modern neural toolkit for sequences, from the gradient pathologies that motivate every architecture, through RNNs, convolutions, and Transformers, into the structured state-space and continuous-time models that now define the frontier, and on to deep forecasting architectures and the pretrained foundation models that forecast zero-shot. The throughline is not a parade of acronyms but a single question asked repeatedly: how should a network carry information across time, and at what cost?
The chapters are ordered to make each architecture feel inevitable. Chapter 9 establishes sequence learning and backpropagation through time, then exposes the long-dependency and gradient problems everything else exists to solve. Chapter 10 answers with RNNs, LSTMs, and GRUs, and reads an RNN as a learned Kalman filter, closing one of the book's central temporal-thread arcs. Chapter 11 offers the convolutional alternative (dilated, causal, WaveNet) and compares the three families head to head. Chapter 12 introduces attention and the Transformer. Chapter 13, a centerpiece, develops structured state-space models (S4, Mamba), linear attention, and Neural ODEs and CDEs, then unifies RNNs, SSMs, and attention into one view. Chapter 14 turns the toolkit toward forecasting specifically (DeepAR, N-BEATS, TFT, PatchTST) and confronts the Transformer-versus-linear debate. Chapter 15 closes with temporal foundation models and LLMs for time series.
The temporal thread is densest here. The Kalman filter and HMM of Part II return as the RNN and neural latent-variable model; ARIMA returns as linear attention; the spectral analysis of Chapter 4 returns inside frequency-domain forecasters. The healthcare running dataset, with its irregular sampling, motivates the Neural CDE of Chapter 13, while finance drives the probabilistic deep forecasters of Chapter 14. By the end of Part III you can build, train, and evaluate a competitive neural forecaster, and you understand why each design choice was made.
Backpropagation through time and the long-dependency gradient pathologies that motivate every later architecture.
LSTMs, GRUs, and seq2seq, with the bridge that reads a recurrent network as a learned Kalman filter.
Dilated, causal convolutions and WaveNet, compared head to head with recurrent and attention-based models.
Self-attention, temporal encodings, and the long-sequence and efficiency techniques that make Transformers practical.
S4 and Mamba, linear attention, and Neural ODEs and CDEs that unify RNNs, SSMs, and attention in one view.
DeepAR, N-BEATS, TFT, and PatchTST, set against the debate over whether Transformers really forecast better.
Pretrained forecasters and LLMs for time series, from Chronos and TimesFM to in-context, zero-shot forecasting.
Where This Part Leads
Part III taught networks to predict sequences; the book now asks what those networks can learn without labels and how to generate sequences of their own. Part IV: Temporal Representation Learning builds on these architectures: the encoders of Part III become the backbones of self-supervised and contrastive representation learning, and the neural latent-variable models foreshadowed here grow into the generative and event models that follow.