"Every morning they ask me where the series is headed, and every morning I answer with the same quiet confidence and a band of uncertainty that grows a little wider with each step I take. Forecast me one day ahead and I am nearly sure of myself. Forecast me a season ahead and I become a soft gray cone of maybe. I am not evasive. I am simply honest about how far my memory can reach, which is exactly as far as the data lets it and not one step more."
A Forecast Interval Widening With Every Step
Chapter Overview
The two previous chapters taught you to look at a time series. Chapter 3 gave you stationarity, the autocorrelation and partial-autocorrelation functions, and the differencing and decomposition that tame a wandering series; Chapter 4 rotated the picture into frequency and let you read off a series' dominant cycles. This chapter finally turns all of that diagnosis into prediction. It builds the classical forecasting models that actually project a series into its future: the moving-average and autoregressive primitives, their combination into ARMA and ARIMA, the seasonal extension SARIMA, the exponential-smoothing and ETS family, and the modern statistical frameworks led by Prophet and the Nixtla ecosystem. Each diagnostic you learned earlier now has a job. The ACF cutoff selects a moving-average order, the PACF cutoff selects an autoregressive order, the differencing that made a series stationary becomes the integration in ARIMA, and the seasonal peaks you found in the spectrum become the seasonal terms of a SARIMA model.
The chapter develops these models in a deliberate order, from parts to wholes. It starts with the two atoms of linear time-series modeling, the moving-average model that carries forward a finite memory of past shocks and the autoregressive model that regresses the present on its own past, and it shows that these are duals: a stationary autoregressive process is an infinite moving average, and an invertible moving-average process is an infinite autoregression. It then fuses them into ARMA, wraps differencing around the result to reach ARIMA for non-stationary series, and threads the Box-Jenkins identification-estimation-diagnostic loop through all of it. SARIMA adds seasonal lag polynomials for data with a fixed period, and SARIMAX lets exogenous drivers enter the model. Exponential smoothing and the ETS state-space taxonomy arrive next as a parallel tradition that often matches or beats ARIMA with less ceremony, and Prophet and statsforecast round out the chapter with the decomposable, automation-friendly tools that dominate industrial practice.
Running underneath every model is one lesson the chapter never lets you forget: a simple, well-validated classical model is a strong baseline, and a strong baseline is the bar a fancier model has to clear. The deep architectures of Part III are powerful, but the M-competitions and a decade of practice keep returning the same uncomfortable verdict, that a carefully fitted ARIMA or ETS, validated by an honest rolling backtest against a seasonal-naive benchmark, beats a great many elaborate models on a great many real series. The final section makes this discipline explicit, weighing fit against complexity with the AIC, AICc, and BIC information criteria, checking residual whiteness with the Ljung-Box test, and ranking candidates by backtested MASE rather than by in-sample fit. The chapter teaches you to forecast and, just as firmly, to distrust a forecast you have not validated.
The temporal thread runs straight through this chapter and out the far side of the book. The autoregressive model is a linear recurrence: the next value is a fixed linear function of a finite window of past values. That single idea generalizes, almost without modification, into the recurrent networks of Chapter 10, where the linear update becomes a learned nonlinear one, and into the structured state-space models of Chapter 13, where ARIMA itself reappears as a form of linear attention. When you write down the AR recurrence in this chapter you are writing, in its simplest and most interpretable form, the equation that the rest of the book spends thirty chapters learning to make deeper. Following the running-dataset convention for Part II, the worked examples here lean on a finance-flavored series, the prices and returns whose trends, volatility, and seasonality exercise every model in turn.
Prerequisites
This chapter assumes the two chapters before it. From Chapter 3: Statistical Foundations you need stationarity, the autocorrelation and partial-autocorrelation functions, white noise, and the differencing and decomposition that make a series stationary, because every model here is identified by reading those diagnostics and ARIMA's integration is exactly that differencing. From Chapter 4: Frequency-Domain and Spectral Analysis you need the idea that a series carries cycles at definite frequencies, since the seasonal peaks of the spectrum are the seasonal terms a SARIMA model fits and the Fourier seasonality Prophet uses. Just as important is the evaluation discipline of Chapter 2: Temporal Data Engineering: forecasting is where leakage is most tempting and most punishing, so the strict separation of past from future, the rolling-origin split, and the refusal to peek across the forecast boundary are assumed throughout and made concrete in this chapter's lab. The mathematics stays light, comfort with the lag operator, geometric series, and ordinary least squares is enough, and readers wanting a refresher on any of these will find the relevant appendices and earlier chapters indexed in the Table of Contents.
If you keep one idea from this chapter, keep this: a low-order ARIMA or ETS, validated by a rolling-origin backtest against a seasonal-naive baseline, is the bar every fancier model must clear. The models in this chapter are cheap to fit, fast to run, and interpretable to the last coefficient, and on a large fraction of real series they are also the most accurate thing you will try. Before reaching for a deep network, fit the simple classical model, backtest it honestly with MASE, and confirm its residuals are white. If a heavier model cannot beat that number on held-out data, the heavier model is not better, it is just more expensive. Every chapter after this one is, in part, an attempt to clear this bar.
Chapter Roadmap
- 5.1 Moving Average Models Separating the moving-average smoother from the MA(q) stochastic model, the always-stationary MA process and its sharp ACF cutoff, invertibility and the infinite-autoregression representation, conditional-sum-of-squares and maximum-likelihood estimation, and mean-reverting forecasts built from scratch and through statsmodels.
- 5.2 Autoregressive Models The AR(p) model as regression on the past, the PACF order signature, stationarity through the characteristic equation and the unit-root boundary, Yule-Walker, least-squares, and maximum-likelihood estimation, and recursive multi-step forecasts that decay back to the mean, with the AR recurrence flagged as the seed of the recurrent networks to come.
- 5.3 ARMA and ARIMA Combining autoregression and moving averages in lag-operator form, integrating a non-stationary series by differencing, the Box-Jenkins identification-estimation-diagnostic loop, forecasting with prediction intervals, and automatic order selection with auto_arima on a non-stationary price series.
- 5.4 SARIMA and Seasonal Modeling Why plain ARIMA fails on seasonal data, the multiplicative SARIMA model and seasonal differencing, reading seasonal ACF and PACF to choose orders, SARIMAX with exogenous regressors, and the multiple-seasonality limit that motivates TBATS and Fourier terms.
- 5.5 Exponential Smoothing and ETS Simple exponential smoothing and its geometrically decaying memory, Holt's linear and damped trend, Holt-Winters seasonality, the ETS state-space taxonomy with AIC-based selection, and the equivalence between simple smoothing and ARIMA(0,1,1).
- 5.6 Prophet and Modern Statistical Frameworks Prophet's decomposable trend-plus-seasonality-plus-holiday model with changepoints and Fourier seasonality, the Prophet-versus-ETS critique, the Nixtla statsforecast ecosystem, and global-versus-local models with hierarchical reconciliation.
- 5.7 Model Selection, Diagnostics, and Information Criteria Balancing fit against complexity with AIC, AICc, and BIC, residual diagnostics for whiteness, normality, and heteroskedasticity, rolling-origin backtesting with MASE and sMAPE, and a principled workflow that selects forecasters by backtested error with information criteria as a tie-breaker.
Once you have worked through the seven sections, the Hands-On Lab below chains them into a single forecasting bake-off. Each lab step maps to a section, so the lab doubles as a review: by the time the harness names a winner you will have fit the ARIMA family of Section 5.3 and Section 5.4, the ETS model of Section 5.5, and the Prophet model of Section 5.6, then ranked them all by the backtested MASE and residual diagnostics of Section 5.7 against the seasonal-naive baseline every serious forecast must beat.
Hands-On Lab: Build a Forecasting Bake-Off
Objective
Run a fair, honest forecasting competition on a single seasonal series from the running dataset and let the data pick the winner. You will fit four contenders, an automatically ordered ARIMA, an automatically selected ETS, a Prophet model, and a humble seasonal-naive baseline, then judge them not by how well they fit the training data but by how well they predict data they never saw. The judging is a rolling-origin backtest scored with the mean absolute scaled error (MASE), the one accuracy metric that is comparable across series and that bakes the seasonal-naive baseline directly into its denominator, so a MASE below one literally means "better than seasonal-naive." For the winning model you will confirm its residuals are white with a Ljung-Box test, because a model whose residuals still carry structure has left forecastable signal on the table, and you will report the winner with calibrated prediction intervals rather than a single brittle point forecast. This is the exact workup a forecasting team runs before it commits a model to production: not "which model fits best" but "which model, validated the way the future will validate it, earns the right to ship."
What You'll Practice
- Fitting an automatically ordered ARIMA and reading its differencing and seasonal terms, following Section 5.3 and Section 5.4.
- Fitting an automatically selected ETS model and recognizing where exponential smoothing matches or beats ARIMA, following Section 5.5.
- Fitting a Prophet model with seasonality and comparing it honestly against the simpler classical tradition, following Section 5.6.
- Scoring every model with a rolling-origin backtest and MASE, and ranking by held-out error rather than in-sample fit, the core discipline of Section 5.7.
- Confirming residual whiteness with the Ljung-Box test and reporting the winner with prediction intervals, the diagnostic close of Section 5.7.
Setup
You need statsmodels for ETS and the classical ARIMA backbone, pmdarima for automatic ARIMA order selection, and prophet for the decomposable model; statsforecast is an optional faster route to the same auto-ARIMA and auto-ETS. A small monthly or daily seasonal series ships with the lab so it runs out of the box, but the bibliography links the M4 and M5 competition data and the Monash archive when you want a genuine many-series benchmark. The seasonal period is fixed and known (twelve for monthly data), so the seasonal-naive baseline simply repeats the value from one full season ago.
pip install statsmodels pmdarima prophet statsforecastSteps
Step 1: Fit an automatically ordered ARIMA
Fit an ARIMA whose orders are chosen automatically by stepwise search, following the Box-Jenkins loop of Section 5.3 and the seasonal extension of Section 5.4. The auto_arima search runs a unit-root test to pick the differencing order, then searches over the autoregressive and moving-average orders (seasonal and non-seasonal) to minimize AICc, so you get a defensible model without hand-tuning every term. Inspect the chosen order: a non-zero seasonal term confirms the series carries the period you expect.
import pmdarima as pm
def fit_auto_arima(y_train, m=12):
model = pm.auto_arima(y_train, seasonal=True, m=m, # m = seasonal period
stepwise=True, suppress_warnings=True,
error_action="ignore")
return model # .predict gives point + interval forecasts
Step 2: Fit an automatically selected ETS
Fit an exponential-smoothing model whose error, trend, and seasonal components are chosen by information criterion, following the ETS taxonomy of Section 5.5. ETS and ARIMA are different traditions that often land in the same neighborhood, so fitting both and comparing them on held-out error is exactly the kind of cheap, informative bake-off this lab is built around. Let the model decide between additive and multiplicative seasonality rather than fixing it by hand.
from statsmodels.tsa.exponential_smoothing.ets import ETSModel
def fit_ets(y_train, m=12):
model = ETSModel(y_train, error="add", trend="add", # damped trend + seasonality
seasonal="add", damped_trend=True,
seasonal_periods=m)
return model.fit(disp=False) # .get_prediction gives intervals
Step 3: Fit a Prophet model
Fit Prophet's decomposable trend-plus-seasonality model, following Section 5.6. Prophet trades the autocorrelation machinery of ARIMA for an explicit additive decomposition with automatic changepoints and Fourier seasonality, which makes it forgiving on messy business series with holidays and missing days. Feed it the same training window as the other contenders so the comparison stays fair, and let it produce its own uncertainty intervals.
from prophet import Prophet
def fit_prophet(df_train): # df has columns ds (date), y (value)
model = Prophet(yearly_seasonality=True, # Fourier seasonality, auto changepoints
weekly_seasonality=False,
interval_width=0.9) # 90% prediction intervals
model.fit(df_train)
return model
Step 4: Score everything with a rolling-origin backtest and MASE
Now judge the contenders the way the future will judge them, following Section 5.7. A rolling-origin backtest repeatedly trains on data up to a cutoff, forecasts the next horizon, records the error, then rolls the cutoff forward, so each model is scored on many genuine out-of-sample windows rather than one lucky split. Score with MASE, which divides the model's mean absolute error by the in-sample mean absolute error of the seasonal-naive baseline, giving a number that is comparable across series and where below one means "beats seasonal-naive." Never let any model see data past its cutoff: this is where the leakage discipline of Chapter 2 earns its keep.
import numpy as np
def mase(y_true, y_pred, y_train, m=12):
naive_err = np.mean(np.abs(y_train[m:] - y_train[:-m])) # seasonal-naive in-sample error
return np.mean(np.abs(y_true - y_pred)) / naive_err # < 1 means better than naive
Step 5: Confirm whiteness and report the winner with intervals
Pick the model with the lowest backtested MASE, then earn the right to ship it by confirming its residuals are white, following the diagnostic close of Section 5.7. The Ljung-Box test asks whether the residual autocorrelations up to a chosen lag are jointly indistinguishable from zero; a large p-value means the residuals look like white noise and the model has extracted the forecastable structure, while a small p-value means signal is still leaking into the errors and the model is not finished. Report the winner as a point forecast wrapped in a prediction interval, never as a bare number, because a forecast without a stated uncertainty is a forecast you cannot act on responsibly.
from statsmodels.stats.diagnostic import acorr_ljungbox
def residuals_are_white(resid, lags=12, alpha=0.05):
lb = acorr_ljungbox(resid, lags=[lags], return_df=True)
pval = lb["lb_pvalue"].iloc[0]
return pval > alpha, pval # True => residuals look like white noise
Expected Output
Running the bake-off prints a small table: each of the four contenders with its rolling-origin MASE, the seasonal-naive baseline pinned at a MASE of one by construction, and the winner highlighted. On a typical seasonal business series the auto-ARIMA and the auto-ETS finish close together, both comfortably below one, with Prophet competitive but rarely dominant unless the series has strong holiday effects, and all three beating the seasonal-naive baseline (any model that does not is disqualified). The winning model's Ljung-Box p-value comes back above the threshold, confirming its residuals carry no remaining autocorrelation, and the final line reports the forecast as a point path with a ninety percent interval that widens with the horizon, exactly the cone the chapter's epigraph describes. The lesson lands in the table itself: the elaborate model has to earn its place against a baseline a child could compute, and on this series the disciplined classical models clear that bar with room to spare.
The lab fits each model with its own library so you see what each tradition does, but Nixtla's statsforecast collapses the entire comparison into a handful of lines. A single StatsForecast object holds AutoARIMA, AutoETS, and SeasonalNaive together, its cross_validation method runs the rolling-origin backtest for all of them at once, and it is fast enough to do this across thousands of series in parallel, which is exactly how the M-competition leaderboards are produced. The roughly fifty lines here become about ten, with the library handling the windowing, the per-series fitting, and the metric bookkeeping. Build the bake-off by hand once so you understand what MASE measures and why the rolling origin is non-negotiable, then lean on statsforecast whenever you need to score many models across many series under a deadline.
Stretch Goals
- Add a SARIMAX contender that ingests an exogenous driver (a calendar or price covariate) and check whether the extra information actually lowers the backtested MASE or merely overfits, the SARIMAX caution of Section 5.4.
- Replace the single MASE with both MASE and sMAPE and confirm the ranking is stable across metrics; where it is not, the disagreement is itself a finding, following Section 5.7.
- Hold the winning model fixed and re-run the backtest on a second series from the running dataset to see whether the same model wins twice, the first hint of the global-versus-local question that Section 5.6 raises and Part III answers.
What's Next?
This chapter forecast a single series from its own past, the univariate case, and that is where most forecasting begins and a great deal of it honestly ends. But series rarely live alone. Sales move with prices, demand moves with weather, one sensor moves with its neighbors, and the future of one variable is often written in the present of another. Chapter 6: Multivariate and Econometric Models opens that door, generalizing the autoregression of this chapter to the vector autoregression that lets several series predict one another, adding the cointegration and error-correction machinery that handles series which drift together, and bringing in the GARCH family that models the changing volatility a returns series carries. The single-series workhorses you built here remain the baselines those richer models must beat, and the rolling-origin discipline you practiced in the lab carries over without a single change.
Bibliography & Further Reading
Foundational Texts
Box, G. E. P., Jenkins, G. M., Reinsel, G. C., Ljung, G. M. "Time Series Analysis: Forecasting and Control," 5th edition. Wiley, 2015. wiley.com
The book that defined the ARIMA family and the identification-estimation-diagnostic loop carried through Sections 5.1 to 5.4; the original source of the Ljung-Box test the lab uses to check residual whiteness.
Hyndman, R. J., Khandakar, Y. "Automatic Time Series Forecasting: The forecast Package for R." Journal of Statistical Software 27(3), 2008. jstatsoft.org
The paper behind auto.arima and the automated ETS selection that Section 5.3 and Section 5.5 describe and the lab automates; the methodological basis of every "auto" model in the bake-off.
Key Books
Hyndman, R. J., Athanasopoulos, G. "Forecasting: Principles and Practice," 3rd edition (FPP3). OTexts, 2021. otexts.com/fpp3
The definitive free textbook for this chapter; its ARIMA and exponential-smoothing chapters are the closest companions to Sections 5.3 through 5.5, and its account of MASE underpins the lab's scoring.
Hyndman, R. J., Koehler, A. B., Ord, J. K., Snyder, R. D. "Forecasting with Exponential Smoothing: The State Space Approach." Springer, 2008. link.springer.com
The complete state-space treatment of exponential smoothing and the ETS taxonomy of Section 5.5, including the likelihood-based model selection the lab's auto-ETS step relies on.
Papers
Taylor, S. J., Letham, B. "Forecasting at Scale." The American Statistician 72(1), 2018 (Prophet). peerj.com/preprints/3190
The Prophet paper: a decomposable trend-plus-seasonality-plus-holiday model with automatic changepoints, the framework Section 5.6 develops and the lab fits as its third contender.
Makridakis, S., Spiliotis, E., Assimakopoulos, V. "The M4 Competition: 100,000 Time Series and 61 Forecasting Methods." International Journal of Forecasting 36(1), 2020. sciencedirect.com
The large-scale competition whose results ground the chapter's recurring lesson: simple, well-validated statistical models and their combinations are formidable baselines, the verdict Section 5.7 builds its workflow around.
Makridakis, S., Spiliotis, E., Assimakopoulos, V. "The M5 Competition: Background, Organization, and Implementation." International Journal of Forecasting 38(4), 2022. sciencedirect.com
The hierarchical retail-demand competition that sharpened the global-versus-local and reconciliation questions of Section 5.6 and reaffirmed strong, cheap baselines on real business data.
Tools & Libraries
statsmodels: time-series analysis in Python (ARIMA, SARIMAX, ETS, Ljung-Box). statsmodels.org
The library behind the from-scratch and library worked examples of Sections 5.1 to 5.5 and the lab's ETS fit and Ljung-Box residual check; the reference implementation of the classical models.
pmdarima: auto_arima and the Box-Jenkins workflow for Python. alkaline-ml.com/pmdarima
Brings R's auto.arima stepwise order selection to Python; the engine of the lab's Step 1 and the automatic identification described in Sections 5.3 and 5.4.
Nixtla statsforecast: fast, vectorized statistical forecasting (AutoARIMA, AutoETS, cross-validation). github.com/Nixtla/statsforecast
The high-performance ecosystem of Section 5.6; its cross_validation method runs the whole rolling-origin bake-off across many series at once, the library shortcut the lab points to.
Prophet: decomposable forecasting from Meta. facebook.github.io/prophet
The official Prophet library and documentation for the trend, seasonality, changepoint, and holiday model of Section 5.6 and the lab's third contender.
sktime: a unified scikit-learn-style interface for time-series forecasting and classification. sktime.net
A consistent fit-predict API wrapping ARIMA, ETS, Prophet, and rolling-origin evaluation; a convenient harness for the model-selection workflow of Section 5.7.
Darts: a Python library for forecasting with a unified model interface. unit8co.github.io/darts
Wraps the classical ARIMA, ETS, and Prophet models alongside deep forecasters under one API with built-in backtesting; a smooth bridge from this chapter's baselines to the deep models of Part III.
Datasets & Benchmarks
Godahewa, R., Bergmeir, C., Webb, G. I., Hyndman, R. J., Montero-Manso, P. "Monash Time Series Forecasting Archive." NeurIPS Datasets and Benchmarks, 2021. forecastingdata.org
A large curated collection of evenly clocked series across many domains; the multi-series benchmark on which the bake-off of this chapter is most informative when scaled up.