"I started life as one tidy autoregression, predicting a single column from its own past, and then they handed me a second column, and a third, and a hundredth, and asked me to keep track of who leans on whom. Now I am a covariance matrix with too many friends, every entry whispering that this series nudges that one, that two of them have quietly promised never to drift far apart, and that the whole crowd is being shoved around by a few shadowy factors I have never actually seen. I have stopped pretending I can model each of you alone. You move together, so I will, at last, model you together."
A Covariance Matrix With Too Many Friends
Chapter Overview
Chapter 5 forecast a single series from its own past, and that univariate world is where most forecasting begins. Real systems, though, are crowds of series that move together. An equity index drags its sector along; an interest rate pulls inflation and exchange rates after it; one industrial sensor rises when its neighbor does; a patient's heart rate and blood pressure trace the same underlying state. Modeling each column alone throws away exactly the information that the other columns carry, namely how the present of one variable foretells the future of another. This chapter generalizes the linear time-series toolkit from one dimension to many, and in doing so it crosses from time-series statistics into econometrics, the discipline that finance, macroeconomics, and multi-sensor engineering rely on to reason about interacting series.
The chapter builds the multivariate toolkit in four movements. It opens with the vector autoregression, the natural generalization of the AR model in which every series is regressed on the recent past of every series at once, giving a single model of the system's joint dynamics; VARMA and high-dimensional regularized variants follow. It then asks what such a model means: Granger causality formalizes the predictive (never interventional) claim that one series helps forecast another, and impulse-response analysis traces how a shock to one variable ripples through the rest. The third movement turns to the long run. A naive regression of one random walk on another manufactures spurious significance, yet some integrated series genuinely share a stable long-run equilibrium; cointegration detects that shared equilibrium and the error-correction model fits the speed at which the system snaps back toward it. The fourth movement confronts dimensionality and variance. Dynamic factor models explain hundreds of co-moving series with a handful of latent common factors, and the ARCH, GARCH, and stochastic-volatility family models the time-varying conditional variance that a returns series carries when calm and turbulent periods cluster.
These are the workhorses of applied econometrics, and they remain in daily production use at central banks, trading desks, and monitoring systems. They are also interpretable to the last coefficient: a VAR coefficient is a lag effect you can read, an error-correction term is a speed of adjustment you can quote, a GARCH parameter is a volatility persistence you can defend. Following the Part II convention, the worked examples lean on a finance-flavored multivariate panel, the prices, returns, and volatilities whose co-movement, shared trends, and clustering volatility exercise every model in the chapter, with the multi-sensor framing surfacing where it sharpens the intuition.
The temporal thread runs straight through this chapter. The dynamic factor model says that many observed series are noisy projections of a few latent drivers evolving over time, which is precisely the premise of the learned temporal representations of Chapter 16 and the latent-variable generative models of Chapter 17; its latent factors prefigure learned embeddings. The volatility models say that uncertainty itself has dynamics, that the spread of tomorrow's distribution depends on today's shocks, which is the seed of the probabilistic deep forecasters of Chapter 19; conditional variance prefigures a forecaster that predicts a full distribution rather than a point. When you fit a factor model or a GARCH here, you are writing, in interpretable closed form, an equation the later chapters learn to make deep.
Prerequisites
This chapter stands on two earlier ones. From Chapter 5: Univariate Forecasting Models you need the autoregressive model, recursive multi-step forecasting, lag-operator notation, and information-criterion order selection, because the vector autoregression is the AR model written for a vector and SARIMA-style intuition carries straight over. From Chapter 3: Statistical Foundations you need stationarity, unit roots and the idea of an integrated series, the autocorrelation function, and white noise, because cointegration is a statement about combining unit-root series into a stationary one, and a VAR is only well behaved when its companion form is stationary. The leakage discipline of Chapter 2: Temporal Data Engineering remains in force throughout, since multivariate models multiply the ways a future value can leak into a feature. The mathematics is heavier here than in Chapter 5, comfort with matrices, eigenvalues, and ordinary least squares is enough, and readers wanting a refresher on any prerequisite will find the relevant chapters and appendices indexed in the Table of Contents.
If you keep one idea from this chapter, keep this: differencing makes a series stationary but throws away its long-run relationships; cointegration keeps those relationships, and latent factors tame the dimensionality. The vector autoregression lets many series predict one another and gives you Granger causality and impulse responses for free. But blindly differencing every series to force stationarity destroys the shared equilibria that often carry the most durable signal, which is what the error-correction model preserves. And when the panel grows to hundreds of series, a full VAR drowns in parameters, while a dynamic factor model recovers the few common drivers behind the crowd. Add the ARCH and GARCH family to model the variance that clusters in time, and you have the econometrician's complete answer to series that move together.
Chapter Roadmap
- 6.1 Vector Autoregression (VAR) and VARMA Generalizing autoregression to many interacting series with the VAR(p) model in vector-matrix form, companion-form stationarity, equation-by-equation OLS estimation, AIC and BIC lag selection, recursive joint forecasting with growing error variance, the identification difficulties of VARMA, and regularized high-dimensional VARs, fit from scratch and through statsmodels.
- 6.2 Granger Causality and Impulse-Response Analysis The predictive rather than interventional meaning of Granger causality and its VAR F-test, the testing procedure and its pitfalls, impulse-response functions through the moving-average representation, and orthogonalization by Cholesky decomposition with forecast-error variance decomposition, run pairwise from scratch and via statsmodels.
- 6.3 Cointegration and Error-Correction Models Why regressing one random walk on another manufactures spurious significance, how a stationary linear combination of integrated series encodes a shared long-run equilibrium, the Engle-Granger and Johansen tests, the vector error-correction model and its speed-of-adjustment coefficients, the Granger representation theorem, and a mean-reverting pairs spread.
- 6.4 Dynamic Factor Models How a handful of latent common factors can drive hundreds of co-moving series and escape the parameter explosion of high-dimensional VAR, the measurement and transition equations, the connection to PCA and state-space filtering, principal-components and EM/Kalman estimation, choosing the number of factors, factor-augmented forecasting and FAVAR, mixed-frequency nowcasting, and the bridge to learned representations.
- 6.5 Volatility Modeling: ARCH, GARCH, and Stochastic Volatility Why financial returns are uncorrelated yet not independent, volatility clustering and heavy tails, the ARCH and GARCH conditional-variance models, the GARCH(1,1) workhorse and its leverage-aware variants, latent stochastic volatility and its state-space connection, Value-at-Risk and the case for probabilistic forecasting, fit from scratch and through the arch library.
Once you have worked through the five sections, the Hands-On Lab below chains them into a single multivariate workup. Each lab step maps to a section, so the lab doubles as a review: by the time you finish you will have fit the VAR of Section 6.1, run the Granger tests and impulse responses of Section 6.2, tested and traded the cointegrated spread of Section 6.3, extracted the common factors of Section 6.4, and fit the GARCH volatility model of Section 6.5, the full econometric pass over a panel of interacting series.
Hands-On Lab: A Multivariate Workup
Objective
Run the complete econometric workup that a quantitative team performs on a panel of interacting series, the multivariate analogue of Chapter 5's bake-off. You will start with several co-moving series, fit a vector autoregression and use it to forecast them jointly, then interrogate that model: which series Granger-cause which, and how does a shock to one ripple through the rest, read from an impulse-response function. You will then take a candidate pair that looks like it shares a long-run equilibrium, test it for cointegration, and if the test passes, build the stationary mean-reverting spread that a pairs-trading desk would actually trade. Widening your view, you will extract a few common factors from a larger panel and confirm they explain most of the co-movement that a per-series model would miss. Finally you will turn to the residual volatility of one series and fit a GARCH model to it, because the conditional variance clusters in time and a single number for the spread is a lie the data will not let you tell. This is the exact sequence an econometrician runs before quoting a joint forecast, a hedge ratio, or a risk number: model the joint dynamics, name the causal-predictive structure, preserve the long-run equilibria, compress the dimensionality, and respect the time-varying variance.
What You'll Practice
- Fitting a VAR with information-criterion lag selection and producing a joint multi-step forecast, following Section 6.1.
- Running pairwise Granger-causality tests and reading an impulse-response function and variance decomposition, following Section 6.2.
- Testing a candidate pair for cointegration and constructing a stationary, mean-reverting spread, following Section 6.3.
- Extracting common factors from a wider panel and confirming they capture most of the shared variation, following Section 6.4.
- Fitting a GARCH(1,1) to the residual volatility of one series and reading its persistence, following Section 6.5.
Setup
You need statsmodels for the VAR, Granger tests, impulse responses, the Johansen and Engle-Granger cointegration tests, and the dynamic factor model, plus the arch library for GARCH and scikit-learn for the principal-components route to the factors. A small multivariate panel of finance-flavored series ships with the lab so it runs out of the box, and the bibliography links the FRED-MD macro panel when you want a genuine high-dimensional benchmark. Every step respects the leakage discipline of Chapter 2: fit on the training window, evaluate on the held-out future, and never let a future value enter a feature.
pip install statsmodels arch scikit-learn pandasSteps
Step 1: Fit a VAR and forecast the system jointly
Fit a vector autoregression on several interacting series, letting an information criterion choose the lag order, following Section 6.1. The VAR regresses every series on the recent past of every series, so a single fitted object forecasts the whole system at once, with each forecast's error variance growing as the horizon lengthens. Check the chosen lag and confirm the fitted system is stationary by verifying the companion matrix eigenvalues lie inside the unit circle.
from statsmodels.tsa.api import VAR
def fit_var(df_train):
model = VAR(df_train)
order = model.select_order(maxlags=12) # AIC/BIC over lag orders
res = model.fit(order.aic) # fit at the AIC-chosen lag
return res # .forecast gives the joint multi-step path
Step 2: Run Granger tests and an impulse-response function
Interrogate the fitted VAR, following Section 6.2. A Granger-causality test asks whether the past of one series improves the forecast of another beyond that series' own past; remember it is a predictive claim, never a claim of real intervention. The impulse-response function then traces how a one-time shock to one variable propagates through the system over the following periods, orthogonalized by a Cholesky ordering so the shocks are uncorrelated. Read which series lead and how long a shock persists.
irf = res.irf(10) # 10-period orthogonalized impulse responses
irf.plot(orth=True) # shock-to-response paths across the system
from statsmodels.tsa.stattools import grangercausalitytests
gc = grangercausalitytests(df_train[["x", "y"]], maxlag=4) # does x Granger-cause y?
Step 3: Test a pair for cointegration and build a mean-reverting spread
Take a candidate pair that appears to share a long-run equilibrium and test it for cointegration, following Section 6.3. Each series alone may be a non-stationary random walk, yet a particular linear combination of the two can be stationary, and that stationary combination is the spread a pairs-trading desk trades because it reliably reverts to its mean. If the Engle-Granger test rejects the no-cointegration null, form the spread from the cointegrating coefficient and confirm it is mean-reverting.
from statsmodels.tsa.stattools import coint
import numpy as np
t_stat, p_value, _ = coint(s1, s2) # Engle-Granger cointegration test
beta = np.polyfit(s2, s1, 1)[0] # cointegrating (hedge) ratio
spread = s1 - beta * s2 # stationary, mean-reverting if cointegrated
Step 4: Extract common factors from a wider panel
Widen the view to a larger panel and recover its few common drivers, following Section 6.4. When dozens or hundreds of series co-move, a full VAR drowns in parameters, but a handful of latent factors often explain most of the shared variation. Extract the factors with principal components as a fast first pass, check how much of the panel's variance the leading factors capture, and confirm they summarize the co-movement that a per-series model would miss.
from sklearn.decomposition import PCA
pca = PCA(n_components=3) # three candidate common factors
factors = pca.fit_transform(panel_train) # latent drivers behind the panel
explained = pca.explained_variance_ratio_.sum() # share of co-movement captured
Step 5: Fit a GARCH to the residual volatility
Turn to the conditional variance of one series, following Section 6.5. Financial returns are nearly uncorrelated yet far from independent, because their volatility clusters: calm follows calm and turbulence follows turbulence. A GARCH(1,1) model captures exactly this by making tomorrow's variance a function of today's squared shock and today's variance. Fit it to the returns (or the VAR residuals) and read the persistence, the sum of the ARCH and GARCH coefficients, which tells you how slowly volatility shocks decay.
from arch import arch_model
am = arch_model(returns, vol="GARCH", p=1, q=1) # GARCH(1,1) conditional variance
gres = am.fit(disp="off")
persistence = gres.params["alpha[1]"] + gres.params["beta[1]"] # near 1 = long-lived volatility
Expected Output
The workup produces a connected picture rather than a single number. The VAR forecast returns a joint path for every series with prediction bands that widen with the horizon, and its companion eigenvalues confirm a stationary, well-behaved system. The Granger tests flag a small set of directed predictive links (typically the faster-moving series leading the slower ones), and the impulse-response plot shows a shock to one variable decaying back to baseline over several periods. The cointegration test, on a genuinely cointegrated pair, returns a small p-value, and the resulting spread oscillates tightly around a constant mean, visibly mean-reverting where the two raw series wander apart. The three leading factors capture the large majority of the panel's variance, demonstrating that a few latent drivers stand behind the crowd. The GARCH fit returns an alpha-plus-beta persistence close to one, the signature of clustered, slowly decaying volatility, and its conditional-variance path tracks the calm and turbulent regimes in the returns. The lesson lands across the five outputs together: series that move together demand to be modeled together, and each econometric tool names a different facet of that togetherness.
The lab leans on two production libraries so the entire multivariate pass fits in a page of code. statsmodels supplies the VAR class with built-in lag selection, impulse responses, and forecast-error variance decomposition; grangercausalitytests, coint, and coint_johansen for the causality and cointegration steps; a VECM class for the error-correction fit; and DynamicFactor for the factor model with its Kalman-filter estimation. The arch library supplies the full GARCH family (plain GARCH, EGARCH, GJR-GARCH) with maximum-likelihood fitting, Value-at-Risk, and volatility forecasting in a few lines. Implement each piece from scratch once, the VAR by stacked OLS, the spread by regression, the factors by PCA, the GARCH by hand-coded likelihood, so you know what the libraries compute, then reach for statsmodels and arch whenever you run this workup for real.
Stretch Goals
- Replace the static spread of Step 3 with a full vector error-correction model via statsmodels
VECMand read the speed-of-adjustment coefficients that say how fast each series snaps back toward equilibrium, the heart of Section 6.3. - Swap the PCA factors of Step 4 for statsmodels
DynamicFactor, which adds factor dynamics through a Kalman filter, and check whether a factor-augmented forecast beats the plain VAR on held-out error, the FAVAR idea of Section 6.4. - Replace the symmetric GARCH of Step 5 with a leverage-aware GJR-GARCH or EGARCH and confirm that negative shocks raise volatility more than positive ones, the asymmetry Section 6.5 motivates.
What's Next?
This chapter modeled interacting series with observable structure: the VAR's lag coefficients, the cointegrating vector, the factor loadings, the GARCH parameters are all things you fit and read directly. But the factor model already hinted at something deeper, that the observed series are noisy projections of a hidden state evolving in time. Chapter 7: State-Space Models and Filtering makes that hidden state the central object. It develops the linear-Gaussian state-space model and the Kalman filter that estimates the latent state optimally as each observation arrives, the hidden Markov model for discrete latent states, and the particle and ensemble filters that handle nonlinear, non-Gaussian dynamics. The dynamic factor model of this chapter is in fact a state-space model in disguise, fit by exactly the Kalman machinery Chapter 7 builds from first principles, and the Kalman filter is itself the classical ancestor of the recurrent network and the structured state-space model that the deep-learning chapters of Part III will learn. The interacting series stay; the structure simply moves into the latent state.
Bibliography & Further Reading
Foundational Texts
Hamilton, J. D. "Time Series Analysis." Princeton University Press, 1994. press.princeton.edu
The standard graduate econometrics reference for everything in this chapter: VAR, Granger causality, impulse responses, cointegration, and ARCH all receive their canonical textbook treatment here.
Lütkepohl, H. "New Introduction to Multiple Time Series Analysis." Springer, 2005. link.springer.com
The definitive book-length treatment of VAR and VARMA models, lag selection, impulse-response and variance-decomposition analysis, and the VECM, the backbone of Sections 6.1 through 6.3.
Key Books
Tsay, R. S. "Analysis of Financial Time Series," 3rd edition. Wiley, 2010. onlinelibrary.wiley.com
The companion text for the finance-flavored material: GARCH and stochastic volatility, Value-at-Risk, and multivariate volatility, the closest reference for Section 6.5 and the lab's returns panel.
Papers
Granger, C. W. J. "Investigating Causal Relations by Econometric Models and Cross-spectral Methods." Econometrica 37(3), 1969. jstor.org
The paper that defined Granger causality, the predictive-causality notion and its testing procedure that Section 6.2 develops and the lab applies pairwise.
Engle, R. F., Granger, C. W. J. "Co-integration and Error Correction: Representation, Estimation, and Testing." Econometrica 55(2), 1987. jstor.org
The foundational cointegration paper, with the two-step test and the Granger representation theorem linking cointegration to error correction, the spine of Section 6.3.
Johansen, S. "Estimation and Hypothesis Testing of Cointegration Vectors in Gaussian Vector Autoregressive Models." Econometrica 59(6), 1991. jstor.org
The maximum-likelihood approach to cointegration that detects multiple cointegrating relations at once, the Johansen test behind the VECM fit of Section 6.3.
Engle, R. F. "Autoregressive Conditional Heteroscedasticity with Estimates of the Variance of United Kingdom Inflation." Econometrica 50(4), 1982. jstor.org
The original ARCH paper that made conditional variance itself a modeled, time-varying quantity, the starting point of the volatility models in Section 6.5.
Bollerslev, T. "Generalized Autoregressive Conditional Heteroskedasticity." Journal of Econometrics 31(3), 1986. sciencedirect.com
The GARCH generalization of ARCH, introducing the GARCH(1,1) workhorse the lab fits and whose persistence Section 6.5 reads off.
Stock, J. H., Watson, M. W. "Dynamic Factor Models, Factor-Augmented Vector Autoregressions, and Structural Vector Autoregressions in Macroeconomics." Handbook of Macroeconomics, vol. 2A, 2016. sciencedirect.com
The authoritative survey of dynamic factor models and FAVAR, the latent-common-factor framework Section 6.4 builds and the lab's factor step recovers.
Tools & Libraries
statsmodels: vector autoregression, VECM, and dynamic factor models in Python. statsmodels.org
The library behind the worked examples and the lab's VAR, Granger tests, impulse responses, Johansen and Engle-Granger cointegration, VECM, and DynamicFactor; the reference implementation for Sections 6.1 through 6.4.
arch: ARCH, GARCH, and volatility models for Python (Kevin Sheppard). bashtage.github.io/arch
The complete GARCH family with maximum-likelihood estimation, leverage-aware variants, Value-at-Risk, and volatility forecasting; the engine of the lab's Step 5 and the library route of Section 6.5.
Datasets & Benchmarks
McCracken, M. W., Ng, S. "FRED-MD: A Monthly Database for Macroeconomic Research." Federal Reserve Bank of St. Louis. research.stlouisfed.org
A large monthly macroeconomic panel maintained for factor-model and FAVAR research, the high-dimensional benchmark on which the dynamic factor model of Section 6.4 and the lab's factor step are most informative.