Point vs Probabilistic Forecasts

"They asked me for tomorrow's demand and I said four hundred and twelve, crisp and confident, and everyone wrote it down as if it were a fact. Nobody asked how sure I was, which is fortunate, because the honest answer was somewhere between two hundred and six hundred, and on a bad day I would not have ruled out zero."

A Point Forecast Hiding How Little It Actually Knows

Everything in the book up to this chapter has, with few exceptions, returned a point forecast: the ARIMA model of Chapter 5 returned a conditional mean, the deep forecasting architectures of Chapter 14 returned a single predicted trajectory, and the representation models of Part IV produced one embedding-driven prediction per horizon. A point forecast answers the question "what is the single most likely value of $y$?" and stops there. The thesis of this chapter is that for almost every real decision this is the wrong question. The right question is "what is the distribution of $y$, given everything I know?", because it is the distribution, not its center, that a downstream decision consumes. We argued in the opening chapter that the purpose of a temporal model is to support a decision (the decision-making thesis of Section 1.3), and a decision under uncertainty is, by definition, an operation on a distribution. We use the unified notation of Appendix A throughout: $\mathbf{x}$ the conditioning information (past observations and covariates), $y$ the future target, $p(y \mid \mathbf{x})$ the predictive distribution, $\hat{y}$ a point forecast.

Why open Part V here rather than with a specific technique? Because uncertainty quantification is not a single algorithm bolted onto forecasting: it is a reframing of what a forecast is. Once you accept that the object a forecaster should emit is a distribution, the rest of this chapter (proper scoring rules in Section 19.2, quantile and distributional regression in Section 19.3, deep probabilistic models in Section 19.4, conformal prediction in Section 19.5, and the evaluation protocols in Section 19.6) becomes a structured study of how to produce, parameterize, calibrate, and score that distribution. This section installs the conceptual foundation: what a probabilistic forecast is, where its uncertainty comes from, what shapes it can take, and how it relates to the point forecasts you already know how to build.

The competencies this section installs are four. To articulate, with a concrete decision, why collapsing a distribution to a point destroys decision-relevant information. To decompose predictive uncertainty into aleatoric and epistemic components and to say which data or model changes reduce which. To recognize the three interchangeable representations of a probabilistic forecast and convert between them. And to take any point forecaster you own and turn it into a calibrated probabilistic forecaster, both by hand and with one library call. These are the load-bearing skills for the rest of Part V and for the decision-making chapters of Part VI, where a policy acts on a belief, not on a guess.

1. Why a Single Number Is Dangerous: Decisions Need the Distribution Beginner

Consider a warehouse deciding how many units of a product to stock for tomorrow. A point forecast says demand will be 412 units, so a naive policy stocks 412. But stocking decisions are asymmetric: the cost of stocking one unit too few (a lost sale, an unhappy customer, perhaps a contractual penalty) is rarely equal to the cost of stocking one unit too many (the holding cost of unsold inventory). The classical newsvendor solution does not stock the mean demand at all: it stocks a quantile of the demand distribution, specifically the quantile at level $\tau = c_u / (c_u + c_o)$, where $c_u$ is the per-unit underage cost and $c_o$ the per-unit overage cost. If lost sales cost three times as much as holding, the optimal stock is the 75th percentile of demand, which might be 520 units, not 412. The point forecast of 412 is not merely imprecise here: it is answering a question nobody asked. The decision needs a quantile, and a quantile is a property of a distribution that a point forecast has thrown away.

This pattern, a decision that depends on the distribution rather than its center, is the rule and not the exception. Three canonical examples make the point across domains, and each recurs later in the book:

• Inventory and supply chain. The order quantity is a cost-weighted quantile of demand (the newsvendor rule above). Two demand distributions with the same mean but different spread call for different orders; a point forecast cannot tell them apart.
• Financial risk. Value-at-Risk and Expected Shortfall, the quantities a risk desk actually reports and that regulators require, are explicitly a left-tail quantile and a tail mean of the return distribution. The expected return is nearly irrelevant to position sizing; the tail is everything. We return to this with the finance series in Section 19.5 and Section 19.6.
• Capacity and staffing. A call center or a power grid sizes capacity to meet demand with a target service level, say "enough to cover demand 99% of the time". That is a high quantile of the load distribution. Provision to the mean and you are under capacity roughly half the time.

In every case the decision is an operation on the distribution: a quantile, a tail expectation, a probability of exceeding a threshold. A point forecast supplies none of these. The collapse from distribution to point is lossy in exactly the dimension the decision cares about, and worse, it is silently lossy: the point forecast looks like a complete answer, so the missing uncertainty is easy to forget until a tail event arrives and the decision made on the mean turns out to have been a coin flip. Figure 19.1.1 contrasts two predictive distributions that share a mean but imply very different decisions.

It is worth being precise about what "dangerous" means, because the danger is not that the point forecast is biased (it may be perfectly unbiased) but that it is silent about its own reliability. A point forecast of 412 carries no warning that it might be 200 or 600. A decision system that consumes it has no way to distinguish a confident forecast from a near-random one, and so it cannot hedge, cannot size a safety buffer, cannot trigger a human review when the model is unsure. The probabilistic forecast restores exactly this missing channel: it reports not just a value but the model's confidence in that value, and that confidence is what lets a downstream system act differently when the future is uncertain. This is the throughline of the entire chapter.

2. Sources of Uncertainty: Aleatoric vs Epistemic Intermediate

If we are to forecast a distribution, we should understand where its width comes from, because different sources call for different remedies. Predictive uncertainty splits into two qualitatively different parts, and conflating them is one of the most common conceptual errors in applied forecasting.

Aleatoric uncertainty is the irreducible randomness in the data-generating process itself: the part of the future that no amount of data or model improvement can pin down, because the world is genuinely stochastic at the resolution we observe it. The roll of a die, the thermal noise on a sensor, the idiosyncratic component of a stock return: these are aleatoric. This is exactly the predictability limit we studied in Section 1.5: every series has an intrinsic entropy floor below which no forecaster, however good, can drive its error. Aleatoric uncertainty is a property of the problem, not of your model, and collecting more data does not shrink it. It can, however, depend on the input: heteroscedastic noise, where the spread of $y$ grows with the level of $\mathbf{x}$, is aleatoric uncertainty that varies across the input space.

Epistemic uncertainty is the reducible uncertainty in the model itself: uncertainty about which parameters, or which model, is correct, arising because we have only finite data. A model trained on a thousand points is less sure of its parameters than the same model trained on a million; that extra wobble is epistemic. Crucially, epistemic uncertainty shrinks as data grows and is large precisely where data is sparse, namely out-of-distribution inputs, rare regimes, and the far future. It is the uncertainty that more data, a better model, or a wider prior would reduce. The total predictive uncertainty is, loosely, the sum of the two:

where $\boldsymbol{\theta}$ are the model parameters, $\mathcal{D}$ the training data, $p(y \mid \mathbf{x}, \boldsymbol{\theta})$ the noise the model assigns once its parameters are fixed (aleatoric), and $p(\boldsymbol{\theta} \mid \mathcal{D})$ the posterior over parameters (epistemic). A pure point forecaster collapses both integrals: it picks a single $\boldsymbol{\theta}$ (ignoring epistemic uncertainty) and reports the center of $p(y \mid \mathbf{x}, \boldsymbol{\theta})$ (ignoring aleatoric uncertainty). The two together define the predictive distribution this chapter learns to produce.

The decomposition is not academic: it tells you what to do when a forecast is too uncertain. If the uncertainty is mostly aleatoric, you are at the predictability floor and the only honest move is to report the width and design the decision to tolerate it (more data will not help). If it is mostly epistemic, the uncertainty is a symptom of insufficient or unrepresentative data, and collecting more, especially in the sparse region, will tighten the forecast. A practitioner who cannot tell the two apart will either waste effort gathering data against irreducible noise or, worse, trust a confident-looking forecast in a region where the model has never seen data and its epistemic uncertainty is enormous.

3. Forms of a Probabilistic Forecast: Density, Quantiles, Samples Intermediate

A probabilistic forecast can be represented in three interchangeable ways, and a practitioner moves fluidly between them because each is convenient for a different purpose. All three describe the same object, the predictive distribution $p(y \mid \mathbf{x})$, and any one can be converted into the others.

Full predictive density (or CDF). The most complete form: a function $p(y \mid \mathbf{x})$ giving the probability density at every possible value of $y$, or equivalently its cumulative distribution $F(y \mid \mathbf{x}) = \Pr(Y \le y \mid \mathbf{x})$. A Gaussian forecast $\mathcal{N}(\mu(\mathbf{x}), \sigma^2(\mathbf{x}))$ is the canonical parametric example, emitting a mean and a variance per step. The density form supports any downstream functional exactly (any quantile, any tail expectation, any exceedance probability) because it is the whole distribution, but it requires committing to a functional form or a flexible parameterization, the subject of Section 19.4.

Quantiles and intervals. Rather than the whole curve, report a few of its quantiles: the value $q_\tau(\mathbf{x})$ such that $\Pr(Y \le q_\tau \mid \mathbf{x}) = \tau$, for a chosen set of levels $\tau$. A central prediction interval is a pair of quantiles, for example $[q_{0.05}, q_{0.95}]$ for a nominal 90% interval. This is the most decision-aligned form, because the newsvendor quantile, the Value-at-Risk quantile, and the service-level quantile of subsection one are read off directly. Quantile regression (Section 19.3) and conformal prediction (Section 19.5) both target this form without ever committing to a density.

Samples and scenarios. Emit a set of plausible future trajectories drawn from the predictive distribution, $\{y^{(1)}, \dots, y^{(S)}\} \sim p(y \mid \mathbf{x})$. Any functional is then estimated by a Monte Carlo average over the samples: a quantile is an order statistic of the sample, a tail expectation is the mean of the worst draws, an exceedance probability is the fraction of samples above the threshold. Samples are the natural output of generative temporal models (Part IV) and of autoregressive forecasters rolled out stochastically, and they shine for multi-step forecasts where the joint distribution over a whole horizon matters (correlated scenarios, not marginal-per-step intervals), which is why scenario generation dominates in energy and finance.

The three forms trade completeness against convenience: the density is exact but needs a parameterization, quantiles are decision-ready but describe only the levels you asked for, samples are flexible and handle joint multi-step structure but introduce Monte Carlo error that shrinks as $1/\sqrt{S}$. Figure 19.1.2 shows the same forecast in all three representations.

One distinction must be made sharply because it is the single most common source of confusion in this whole topic: a prediction interval is not a confidence interval. A prediction interval is a statement about a future observation: a 90% prediction interval $[\ell, u]$ claims that the realized future value $y$ will fall in $[\ell, u]$ with probability 0.90. A confidence interval is a statement about a parameter: a 90% confidence interval for the mean claims that the interval, computed from data, covers the true (fixed, unknown) mean with probability 0.90 over repeated samples. The prediction interval includes the irreducible aleatoric noise of the future observation and so is always wider than the confidence interval for the mean, which only accounts for the epistemic uncertainty in estimating that mean. A forecaster reports prediction intervals: it is forecasting an observation, not estimating a parameter. Reporting a confidence interval for the conditional mean and calling it a forecast interval is a textbook error that produces intervals far too narrow, because it omits exactly the aleatoric term of subsection two.

4. How Point and Probabilistic Forecasts Relate Intermediate

Point and probabilistic forecasts are not rival species: a point forecast is a summary of a probabilistic one, and which summary is optimal is decided entirely by the loss function you score it with. This is the bridge that connects everything you built in Parts II to IV to everything in Part V, and it is worth stating as a theorem-shaped fact. Given a predictive distribution $p(y \mid \mathbf{x})$ and a scoring loss $L(\hat{y}, y)$, the optimal point forecast is the one that minimizes the expected loss under that distribution:

The remarkable part is that the minimizer is a different functional of the distribution for different losses, and the two losses you have used throughout the book pick out the two most familiar summaries:

• Under squared error $L(\hat{y}, y) = (\hat{y} - y)^2$, the expected loss is minimized at the mean: $\hat{y}^\star = \mathbb{E}[y \mid \mathbf{x}]$. This is why every model trained with MSE (most neural forecasters, OLS, the Kalman filter's update) is, at its optimum, predicting the conditional mean of the predictive distribution.
• Under absolute error $L(\hat{y}, y) = |\hat{y} - y|$, the expected loss is minimized at the median: $\hat{y}^\star = \mathrm{median}(y \mid \mathbf{x})$. This is the MAE-to-median fact we first met when choosing loss functions; it is why MAE-trained models are robust to skew and outliers, because the median ignores tail mass that the mean chases.
• Under the pinball (quantile) loss at level $\tau$, the minimizer is the $\tau$-quantile, which is the precise mechanism that lets a single model emit calibrated quantiles in Section 19.3.

So the loss you train on is not a neutral implementation detail: it silently selects which point of the predictive distribution your "point forecast" reports. Train on MSE and you get the mean; train on MAE and you get the median; if the predictive distribution is skewed (as demand, returns, and durations all are), the mean and median differ, and the "point forecast" you report depends entirely on which loss you happened to choose. This is the deepest reason a point forecast is incomplete: it is one chosen projection of a distribution, and the distribution is the thing that determined the projection in the first place. Reading the relationship in reverse gives the constructive recipe of subsection five: if a point forecast is a summary of a distribution, then to build a probabilistic forecast we put the distribution back around the point, either by modeling the residuals, by predicting distribution parameters directly, or by collecting empirical quantiles of past errors.

5. Worked Example: Turning One Point Forecaster Into Three Probabilistic Ones Advanced

We now make the whole section executable. We take a single point forecaster (a simple linear autoregressive model of the kind built in Chapter 5) and upgrade it to a probabilistic forecaster three ways from scratch, exactly the three constructions subsection four's reverse reading prescribed: residual-based intervals, a Gaussian distributional head, and empirical quantiles of the errors. We then check the empirical coverage of one interval, and finally collapse the entire thing into a few lines of a modern library. Code 19.1.1 builds the data and the point forecaster and produces residuals on a held-out calibration split.

import numpy as np

rng = np.random.default_rng(0)

# Synthetic AR(1)-with-trend series; the noise is the irreducible aleatoric part.
T = 600
eps = rng.normal(0, 2.0, size=T)            # aleatoric noise, sigma = 2.0
y = np.zeros(T)
for t in range(1, T):
    y[t] = 0.6 * y[t-1] + 0.05 * t + eps[t]  # AR(1) + linear trend

# Design matrix for a lag-1 + trend linear forecaster: x_t = [1, y_{t-1}, t].
X = np.column_stack([np.ones(T-1), y[:-1], np.arange(1, T)])
target = y[1:]

# Three-way split: train (fit), calibration (residuals), test (evaluate).
n = len(target)
i_tr, i_cal = int(0.6*n), int(0.8*n)
Xtr, ytr = X[:i_tr], target[:i_tr]
Xcal, ycal = X[i_tr:i_cal], target[i_tr:i_cal]
Xte, yte = X[i_cal:], target[i_cal:]

# Fit the POINT forecaster by ordinary least squares (predicts the conditional mean).
beta, *_ = np.linalg.lstsq(Xtr, ytr, rcond=None)
point = lambda Xq: Xq @ beta               # the single-number forecast

# Calibration residuals: the empirical stand-in for the predictive noise.
resid = ycal - point(Xcal)
print("point RMSE (test) = %.3f" % np.sqrt(np.mean((yte - point(Xte))**2)))
print("calib resid mean = %.3f, std = %.3f" % (resid.mean(), resid.std()))

point RMSE (test) = 2.041
calib resid mean = -0.027, std = 1.985

Now the three from-scratch probabilistic upgrades. Code 19.1.2 implements all three: (a) residual-based intervals add a symmetric band around the point forecast, sized from the residual standard deviation under a Gaussian assumption; (b) a Gaussian head emits an explicit $(\mu, \sigma)$ predictive density per point; (c) empirical quantiles of the residuals make no distributional assumption and read the interval straight off the residual order statistics. Each turns the one number from Code 19.1.1 into a distribution, exactly the "put the distribution back around the point" recipe of subsection four.

from scipy.stats import norm

mu_te = point(Xte)                          # the shared point forecast (the mean)

# (a) RESIDUAL-BASED Gaussian interval: point +/- z * residual std. 90% -> z = 1.645.
sigma_hat = resid.std()
z = norm.ppf(0.95)                          # one-sided 95% -> central 90%
lo_a, hi_a = mu_te - z*sigma_hat, mu_te + z*sigma_hat

# (b) GAUSSIAN HEAD: emit a full predictive density N(mu, sigma^2) per test point.
#     Here sigma is homoscedastic (constant); a learned head would predict sigma(x).
predictive = [norm(loc=m, scale=sigma_hat) for m in mu_te]
lo_b = np.array([d.ppf(0.05) for d in predictive])   # 5th percentile of each density
hi_b = np.array([d.ppf(0.95) for d in predictive])   # 95th percentile

# (c) EMPIRICAL QUANTILES: no Gaussian assumption; use residual order statistics.
q_lo, q_hi = np.quantile(resid, [0.05, 0.95])         # empirical 5% and 95% residuals
lo_c, hi_c = mu_te + q_lo, mu_te + q_hi               # shift the band onto each forecast

for name, lo, hi in [("residual-Gaussian", lo_a, hi_a),
                     ("Gaussian head    ", lo_b, hi_b),
                     ("empirical quantile", lo_c, hi_c)]:
    width = np.mean(hi - lo)
    print("%s : mean 90%% width = %.3f" % (name, width))

residual-Gaussian  : mean 90% width = 6.530
Gaussian head      : mean 90% width = 6.530
empirical quantile : mean 90% width = 6.479

An interval is only useful if its stated coverage is true, so we check it. Code 19.1.3 computes the empirical coverage of the empirical-quantile interval on the test set, the fraction of realized values that actually fell inside the nominal 90% band, and reports the gap from the 0.90 target. This is the calibration check that Section 19.2 turns into a formal scoring rule.

# Empirical coverage: fraction of test targets inside the nominal-90% interval.
inside = (yte >= lo_c) & (yte <= hi_c)
coverage = inside.mean()
print("nominal coverage  = 0.900")
print("empirical coverage = %.3f  (gap %.3f)" % (coverage, abs(coverage - 0.90)))
print("interval count = %d, covered = %d" % (len(yte), inside.sum()))

nominal coverage  = 0.900
empirical coverage = 0.892  (gap 0.008)
interval count = 120, covered = 107

from statsforecast import StatsForecast
from statsforecast.models import AutoARIMA
import pandas as pd

# Long-format frame the API expects: unique_id, ds (time), y (value).
df = pd.DataFrame({"unique_id": "series_1",
                   "ds": pd.date_range("2020-01-01", periods=len(y)),
                   "y": y})

sf = StatsForecast(models=[AutoARIMA()], freq="D")
sf.fit(df)
# level=[90] returns the 90% PREDICTION interval columns alongside the point forecast.
fc = sf.predict(h=14, level=[90])           # point + lo-90 + hi-90, in one call
print(fc[["AutoARIMA", "AutoARIMA-lo-90", "AutoARIMA-hi-90"]].head())