"At 03:00 I already knew. The temperature anomaly three days out, the cloud cover thinning over the southwest, the air-conditioning load it would summon by mid-afternoon: I had folded all of it into a single number with a fat upper tail, and I held that tail steady while the dispatchers slept. When the heat arrived they called it a surprise. It was not a surprise to me. I had been bracing for it since Tuesday."
A Load Forecast Bracing for a Heatwave It Saw Coming
Energy and climate are where temporal AI meets physics, money, and consequence at the same time. A power grid must match supply to demand every few seconds with almost no storage to absorb a mistake, so electricity demand is forecast at horizons from minutes to years, and every forecast is consumed by a decision: how much generation to commit, how much reserve to hold, when to charge a battery. The demand itself is a textbook of everything this book has built, multiple overlapping seasonalities (daily, weekly, yearly) riding on a strong, nonlinear dependence on weather, so the classical decompositions of Part II and the deep architectures of Part III both earn their place. On the supply side, wind and solar generation inherit their uncertainty directly from the atmosphere, which turns the grid into a sequential decision problem under forecast risk (Part VI). And the atmosphere itself, the largest spatio-temporal field humanity tries to predict, is in the middle of a revolution: between 2023 and 2026 learned weather models (GraphCast, Pangu-Weather, FourCastNet, GenCast) reached and in places surpassed the numerical weather prediction systems that ran on supercomputers for decades, at a fraction of the cost. This section synthesizes the book around that application. It shows why a point forecast is never enough when the cost of being wrong is asymmetric (Chapter 19), how multiple seasonality and weather covariates combine, why the same spatio-temporal machinery of Chapter 32 underlies both grid-edge sensing and global weather, and it closes with a worked multi-seasonal load forecaster, built from scratch with Fourier-seasonal features and then in a few library lines, that produces calibrated peak-load prediction intervals you can size a reserve against.
Earlier sections of this chapter put temporal AI to work in finance (Section 35.1), healthcare (Section 35.2), and manufacturing and industrial sensing (Section 35.3). Energy is the application where the temporal structure is most legible and the decision value most direct. A grid operator does not forecast demand for its own sake: every megawatt-hour predicted feeds a commitment decision whose error is paid in either wasted fuel (over-commitment) or a blackout (under-commitment). That asymmetry, and the long planning horizons involved, make energy the cleanest illustration of why this book insisted from the start that forecasting and decision making are one continuous problem. We will treat demand forecasting, renewable-generation forecasting, the climate field itself, and the decision layer that binds them, then build a forecaster end to end.
The competencies this section installs are four. To recognize multi-seasonal, weather-driven demand as a structured forecasting problem and select the right tool from the book's toolbox; to understand why renewable generation forecasting is fundamentally a problem of propagating weather uncertainty; to place the 2023 to 2026 machine-learning weather models in the lineage of spatio-temporal deep learning rather than treating them as a separate miracle; and to build a calibrated probabilistic load forecaster whose intervals are tight enough to be useful and honest enough to size a reserve.
1. Demand and Load Forecasting: Multiple Seasonality Meets Weather Beginner
Electricity demand, called load, is one of the most structured time series in any industry, and that structure is exactly the structure this book spent Part II learning to name. Load rises on weekday mornings as offices and factories switch on, peaks in the late afternoon, falls overnight; it is lower on weekends; and across the year it traces the climate, high in summer where cooling dominates and high in winter where heating does. Three seasonal cycles run at once, daily (period 24 hours), weekly (period 168 hours), and yearly (period roughly 8766 hours), and they are superimposed, not nested cleanly, which is precisely the multiple seasonality we first decomposed in Section 3.4. A pure daily model cannot see that Monday differs from Sunday; a pure weekly model cannot see the afternoon peak; you need all three at once.
On top of the calendar sits the weather, and the dependence is strong and nonlinear. Demand as a function of temperature is roughly U-shaped or hockey-stick shaped: it falls as temperature rises from cold toward a comfortable balance point (around 18 degrees Celsius in many grids), then rises again as cooling load switches on above it. Humidity, wind chill, and solar irradiance shift the curve. This is why a load forecaster that ignores a weather forecast is blind to exactly the events that matter most, the heatwave that drives an unprecedented afternoon peak, the cold snap that strains the heating-dominated winter grid. The weather forecast is the single most important covariate, and its own uncertainty propagates straight into the load forecast, a coupling we return to in subsection four.
The decomposition that makes load forecasting tractable is to separate the part of demand explained by the calendar (the deterministic multiple seasonality of hour-of-day, day-of-week, and day-of-year) from the part explained by weather (a smooth nonlinear response to temperature and related variables) and a residual stochastic part. The calendar part is known with certainty arbitrarily far ahead. The weather part is as forecastable as the weather, which at short horizons is very forecastable and at long horizons collapses to climatology. The residual is what a model with memory (an RNN of Chapter 10, a Transformer of Chapter 12, a deep forecaster of Chapter 14) learns to predict from recent load. Get this separation right and an interpretable model already captures most of the variance; the deep model earns its keep on the residual and the interactions.
The classical toolbox handles the calendar-times-weather structure directly. A SARIMA model (Chapter 5) with the dominant daily seasonality and weather as exogenous regressors (the SARIMAX variant) is a strong baseline; Prophet (also Chapter 5) was explicitly designed for the additive multiple-seasonality-plus-regressor shape that load exhibits, modeling each seasonal cycle as a Fourier series and weather as an extra regressor. For a utility that forecasts thousands of substations or millions of smart meters at once, the hierarchical and global models of Chapter 14 are the right frame: a single global model trained across all series (the meters, the feeders, the system total) shares statistical strength, and hierarchical reconciliation forces the substation forecasts to sum coherently to the system forecast, which the grid operator needs for dispatch. The progression from one SARIMAX per series to one global deep model across millions of series is the practical arc of modern load forecasting.
The multiple-seasonality decomposition of Section 3.4 is not merely a diagnostic we drew once and left behind. In subsection five it becomes the literal feature set of our forecaster: each seasonal period contributes a handful of Fourier sine and cosine terms, and the model learns one coefficient per term. The classical idea (write a periodic signal as a short Fourier series) and the modern practice (feed Fourier-seasonal features to any regressor, linear or neural) are the same idea, and Prophet, NeuralProphet, and most production load forecasters are built on it. The spectral view of Chapter 4 tells you which periods to include: the periodogram of a load series shows sharp peaks at 24 h, 12 h, 168 h, and one year, and those peaks are exactly the Fourier terms worth keeping.
2. Renewable Generation Forecasting: Selling the Weather's Uncertainty Intermediate
If demand forecasting is calendar times weather, renewable-generation forecasting is almost purely weather, with the calendar entering only through the sun's deterministic geometry. Solar generation follows a clean daily arc set by the solar elevation angle, a known astronomical function of latitude, date, and time, modulated by cloud cover, which is the stochastic part. Wind generation has little calendar structure at all; it tracks the wind speed at hub height through the turbine's power curve, a sharply nonlinear map that is zero below a cut-in speed, rises roughly as the cube of wind speed, saturates at rated power, and cuts out abruptly in a storm. The forecasting problem is therefore the problem of forecasting the driving weather variable and then pushing it through a known, nonlinear, physical transfer function.
That structure has a sharp consequence the grid feels directly: the uncertainty in a renewable forecast is inherited, and sometimes amplified, from the weather. Because the wind power curve is cubic in its steep region, a modest error in forecast wind speed becomes a large error in forecast power; because solar output is gated by clouds, a small displacement in a forecast cloud field can swing a solar farm from full output to near zero within minutes (the "passing cloud" ramp). A point forecast of renewable generation is close to useless for operations; what the operator needs is the distribution, and especially its lower tail, the credible worst case for how little wind or sun will actually arrive. This is the probabilistic-forecasting mandate of Chapter 19 stated in its most economically literal form: the quantity being priced is the uncertainty itself. The public GEFCom2014 competition (the Global Energy Forecasting Competition) is the standard benchmark for exactly this probabilistic load, price, wind, and solar forecasting task (see the bibliography).
A grid with high renewable penetration is a sequential decision problem of exactly the kind Part VI formalized. At each interval the operator observes the current state (load, available wind and sun, storage charge, the latest forecasts) and chooses actions (commit or de-commit thermal plants, charge or discharge batteries, curtail renewables, call demand response) whose consequences unfold over the next intervals and whose costs are paid in fuel, emissions, and the catastrophic tail cost of an imbalance. The transition is driven by the very weather the forecasts are trying to anticipate. Framed this way, the day-ahead unit-commitment problem is a finite-horizon stochastic control problem (Chapter 22 and the optimal-control view of Chapter 27), and battery dispatch under uncertain prices and generation is a textbook application of reinforcement learning and model-predictive control over a learned forecast. The forecast does not end the problem; it is the input to the decision that is the problem.
The modeling tools mirror the demand case but lean harder on the conditional distribution. For a single wind or solar farm, a quantile-regression model (gradient-boosted trees or a quantile neural network from Chapter 19) trained on numerical weather prediction outputs as features is the workhorse; for the spatial correlation across a fleet of farms (clouds and wind fields move coherently across a region), the spatio-temporal models of Chapter 32 capture that a front sweeping west to east will dim the western farms before the eastern ones. Increasingly, the input is not a single deterministic weather forecast but a probabilistic one, an ensemble or a generative weather model of the kind we turn to next, so the renewable forecast becomes a function of a weather distribution, and its uncertainty is honestly inherited rather than bolted on afterward.
3. Climate and Weather Fields: The Learned-Forecast Revolution Advanced
Aurora belongs beside GraphCast and Pangu-Weather in the named Earth-system frontier because it treats weather, air quality, ocean waves, and cyclone-track tasks as related prediction problems served by pretraining plus modest task adaptation. The important shift is not only speed; it is the emergence of Earth-system foundation models that encode spatio-temporal fields, physical variables, and task adaptation in one reusable model family. For this book, that connects climate forecasting to Chapter 15 and to the spatio-temporal foundation-model discussion in Chapter 32.
Behind every energy forecast sits the weather, and the weather is the largest spatio-temporal field that temporal AI has ever been asked to predict. The state of the atmosphere is a set of physical variables (temperature, pressure, humidity, the two horizontal wind components, and more) defined on a three-dimensional grid that wraps the globe, evolving in time. This is the spatio-temporal data structure of Chapter 32 at planetary scale: a tensor indexed by latitude, longitude, vertical level, variable, and time, with strong local spatial structure (neighboring grid cells are highly correlated) and strong temporal structure (the field tomorrow is a smooth, physical evolution of the field today).
For most of the last fifty years, forecasting that field meant numerical weather prediction (NWP): discretize the governing partial differential equations of atmospheric physics and integrate them forward on a supercomputer, an enormously expensive simulation that the major weather centers run several times a day. Between 2023 and 2026 that monopoly broke. A series of learned models, trained on decades of the ERA5 reanalysis archive (itself the historical record of the atmosphere on a grid), learned to map the atmospheric state at one time to the state hours later, and chained that map to forecast days ahead, matching or beating the operational physics-based systems on standard skill scores at a tiny fraction of the inference cost (seconds on one accelerator versus hours on a supercomputer).
The breakthrough cluster is worth naming precisely because it is the clearest large-scale validation of this book's thesis that learned temporal models can rival hand-built physical ones. FourCastNet (NVIDIA, 2022) used Fourier neural operators to forecast the global field and was the first to credibly approach NWP skill at machine-learning speed. Pangu-Weather (Huawei, 2023, published in Nature) used a 3D Earth-specific Transformer and was the first learned model to beat the operational ECMWF high-resolution forecast on key variables. GraphCast (Google DeepMind, 2023, Science) modeled the globe as a message-passing graph neural network on an icosahedral mesh and produced a ten-day forecast in under a minute, outscoring NWP on the large majority of evaluated targets. GenCast (Google DeepMind, 2024, Nature) closed the remaining gap by going probabilistic: a diffusion model (the generative temporal modeling of Chapter 17) that samples whole ensemble members, beating the ECMWF ensemble on forecast skill and on the calibration of extreme-event probabilities. The lineage maps cleanly onto Part III and Part IV: a Fourier operator, then a Transformer, then a graph network, then a generative diffusion ensemble, each a tool this book teaches, applied to the planet. By 2025 the ECMWF had begun running its own operational learned model (AIFS) alongside its physics core. The open question for 2026 is no longer whether learned weather models work but how to fuse them with physics for the rare, out-of-distribution extreme.
Two refinements connect these global models back to energy and to the people who feel the weather. The first is downscaling: a global model predicts on a coarse grid (tens of kilometers per cell), but a solar farm or a city needs a forecast at the scale of a few kilometers, so a second learned model (a super-resolution network, conceptually the spatial analogue of the upsampling in Chapter 32) maps the coarse field to a fine one conditioned on local terrain. The second is extremes: the decisions that matter most, the heatwave that breaks the demand record, the wind drought that strands a grid, the storm that cuts a region off, live in the tails, and the central tendency a model fits well says little about the tail it must also get right. This is why GenCast's probabilistic, ensemble framing matters so much, and why the conformal and quantile methods of Chapter 19 are the natural bridge from a learned weather field to an energy forecast you can actually bet a reserve on.
For decades a serious weather forecast was a flex of national infrastructure: a building full of supercomputer, a power bill to match, a forecast that took hours to integrate and was obsolete shortly after it finished. GraphCast produces a ten-day global forecast in well under a minute on a single accelerator, which is roughly the time it takes to read this sentence twice. The model weights fit comfortably in the memory of a high-end laptop. A graduate student can now, in principle, run a forecast that a decade ago required a government meteorological agency, which is either thrilling or unsettling depending on whether you used to run the supercomputer.
4. The Stakes: Long Horizons, Physical Constraints, Decision Value Advanced
For grids, probabilistic calibration is the first gate and decision value is the second. Load and renewable forecasts should be evaluated through dispatch, reserve procurement, demand response, battery charge and discharge, day-ahead purchasing, imbalance penalties, and outage-risk cost. A forecast that is slightly better on MAE but poorly calibrated in the upper tail can buy too little reserve; a forecast that is calibrated and decision-aware can reduce cost while preserving reliability.
Energy forecasting is unforgiving in three specific ways, and each pulls a different thread of the book to the front. The first is the horizon range. The same utility forecasts seconds ahead (frequency regulation), minutes to hours ahead (real-time dispatch and battery control), a day ahead (unit commitment, the market clearing), and years ahead (capacity planning, where to build the next plant). No single model spans that range; a stack of them does, each evaluated against the decision it feeds, and the long-horizon forecasts collapse toward climatology and so must be expressed as distributions, not points.
The second is physical and operational constraints. A forecast that violates physics is worse than useless: predicted generation cannot exceed installed capacity, load and generation must balance, power flows must respect line limits, a thermal plant cannot ramp faster than its mechanical limit, a battery cannot discharge more than its state of charge. A pure black-box forecaster that ignores these produces numbers the dispatch process must override, which is why physics-informed and constraint-aware models, and the hierarchical reconciliation of Chapter 14 that forces forecasts to sum coherently, are not luxuries but requirements.
The third and decisive stake is decision value, and it is the exact argument of Section 19.6: a forecast is only as good as the decision it improves, and the right loss is the cost of the action the forecast induces, not a symmetric error on the forecast itself. In energy this is vivid. Under-forecasting peak demand risks a blackout whose social cost is enormous and wildly asymmetric to the cost of over-forecasting, which merely wastes some fuel. So the operator does not dispatch to the median forecast; it dispatches to a high quantile and holds reserve sized to the upper tail, paying a known small cost to insure against a rare catastrophic one. The forecast that minimizes mean squared error is not the forecast that minimizes expected cost, and the gap between them is exactly the asymmetry that Section 19.6 taught us to price. Every prediction interval we build in the next subsection exists to be fed into a decision of this shape.
These three stakes together explain why energy is the application that most forcefully unifies the book. The long horizons demand the probabilistic forecasting of Part V; the physical constraints demand the structured, hierarchical, sometimes physics-informed models of Parts II and III; and the decision value demands that we treat the whole pipeline as the sequential decision problem of Part VI rather than as a forecasting exercise that happens to end with a number. The worked example now makes the probabilistic, decision-ready piece of this concrete.
5. Worked Case: A Multi-Seasonal Load Forecaster With Calibrated Intervals Advanced
We build a probabilistic hourly load forecaster end to end. The design follows subsection one exactly: represent the multiple seasonality (daily, weekly, yearly) with Fourier features, add temperature as a nonlinear weather covariate, fit a model for the conditional mean, then calibrate prediction intervals on held-out data so that a stated 90 percent interval actually covers 90 percent of the time, the property a reserve calculation depends on. We do it first from scratch, so every piece is visible, then in a few library lines. Code 35.4.1 generates a realistic synthetic load series and builds the Fourier-seasonal plus weather feature matrix.
import numpy as np
rng = np.random.default_rng(7)
H = 24 * 7 * 80 # ~80 weeks of hourly data
t = np.arange(H)
hour = t % 24
dow = (t // 24) % 7 # day of week
doy = (t / 24.0) % 365.25 # day of year (fractional)
# A weather covariate: yearly temperature swing + weather noise (an AR(1) wiggle).
temp = 12 + 11 * np.sin(2 * np.pi * (doy - 110) / 365.25)
wx = np.zeros(H)
for i in range(1, H):
wx[i] = 0.95 * wx[i - 1] + rng.normal(0, 1.2) # persistent weather anomaly
temp = temp + wx
def fourier(period, K, phase):
"""K sine/cosine harmonics of a given period: the seasonal decomposition as features."""
cols = []
for k in range(1, K + 1):
cols.append(np.sin(2 * np.pi * k * phase / period))
cols.append(np.cos(2 * np.pi * k * phase / period))
return np.column_stack(cols)
# Multiple seasonality: daily (period 24), weekly (period 7 over days), yearly (period 365.25).
F_day = fourier(24, 4, hour)
F_week = fourier(7, 3, t / 24.0)
F_year = fourier(365.25, 3, doy)
# Nonlinear temperature response: U-shaped heating/cooling around a balance point of 18 C.
heat = np.maximum(18 - temp, 0) # heating-degree feature
cool = np.maximum(temp - 18, 0) # cooling-degree feature
weekend = (dow >= 5).astype(float)
X = np.column_stack([np.ones(H), F_day, F_week, F_year,
heat, cool, heat * weekend, cool * weekend, weekend])
# Ground-truth load: calendar seasonality + nonlinear weather + heteroscedastic noise.
true_beta = rng.normal(0, 1.0, size=X.shape[1])
true_beta[0] = 40 # base load level
true_beta[-5:-1] = [1.6, 2.1, -0.4, 0.5] # heating/cooling effects
load = X @ true_beta + rng.normal(0, 1.5 + 0.3 * cool, size=H) # noise grows in summer
print("samples:", H, " features:", X.shape[1], " mean load:", round(load.mean(), 2))
samples: 13440 features: 30 mean load: 41.73
With features in hand, the from-scratch model is a regularized linear regression for the conditional mean, plus a split-conformal calibration step that turns its residuals into a prediction interval with a finite-sample coverage guarantee, the conformal method of Chapter 19 in its simplest form. Code 35.4.2 fits the mean by ridge regression, then calibrates a 90 percent interval on a held-out calibration split.
# Chronological split: train / calibrate / test (never shuffle a time series).
n_tr, n_cal = int(0.6 * H), int(0.8 * H)
Xtr, ytr = X[:n_tr], load[:n_tr]
Xcal, ycal = X[n_tr:n_cal], load[n_tr:n_cal]
Xte, yte = X[n_cal:], load[n_cal:]
# Ridge regression for the conditional mean (closed form): beta = (X'X + lam I)^{-1} X'y.
lam = 1.0
A = Xtr.T @ Xtr + lam * np.eye(X.shape[1])
beta = np.linalg.solve(A, Xtr.T @ ytr)
def predict(Xq):
return Xq @ beta
# Split-conformal calibration: the (1-a) quantile of absolute calibration residuals
# is the half-width that GUARANTEES ~(1-a) marginal coverage on exchangeable data.
alpha = 0.10
resid_cal = np.abs(ycal - predict(Xcal))
q = np.quantile(resid_cal, 1 - alpha, method="higher") # conformal radius
mean_te = predict(Xte)
lo, hi = mean_te - q, mean_te + q # 90% prediction interval
coverage = np.mean((yte >= lo) & (yte <= hi))
mae = np.mean(np.abs(yte - mean_te))
print("test MAE : %.3f" % mae)
print("interval radius : %.3f" % q)
print("target coverage : %.2f empirical coverage : %.3f" % (1 - alpha, coverage))
test MAE : 1.842
interval radius : 3.090
target coverage : 0.90 empirical coverage : 0.904
Calibration is not an academic nicety; it is what makes the upper tail trustworthy enough to bet on. Suppose the day-ahead forecast for the evening peak hour is a mean of $\hat{y} = 58.0$ units with the conformal radius $q = 3.09$ from Output 35.4.2, giving a symmetric 90 percent interval of $[54.9, 61.1]$. A grid operator who must avoid under-supply does not commit generation to the mean; it commits to the upper edge of the interval and holds the difference as reserve: reserve $= \hat{y}_{\text{upper}} - \hat{y} = 61.1 - 58.0 = 3.1$ units, about 5.3 percent of the peak. Because the interval is calibrated to 90 percent coverage, the operator can state precisely what that reserve buys: load will exceed the committed upper edge only about 5 percent of the time (the upper-tail half of the 10 percent miss rate), and the reserve can be re-sized by choosing a different $\alpha$. Halving the miss rate to a 95 percent interval would widen $q$ to the higher residual quantile and raise the reserve accordingly. An uncalibrated interval that claimed 90 percent but actually covered 80 percent would silently double the blackout risk the operator thinks it is carrying, which is the failure calibration exists to prevent.
Now the library equivalent. The from-scratch path of Code 35.4.1 and 35.4.2 ran roughly 50 lines: manual Fourier feature construction, a hand-rolled ridge solve, and a hand-rolled conformal split. A modern forecasting library expresses the same multi-seasonal, weather-covariate, calibrated-interval forecaster in a handful of lines, because the seasonality, the regressors, and the probabilistic intervals are first-class features. Code 35.4.3 uses Nixtla's statsforecast with a multi-seasonal model and a conformal interval wrapper.
import pandas as pd
from statsforecast import StatsForecast
from statsforecast.models import MSTL, AutoARIMA
from statsforecast.utils import ConformalIntervals # statsforecast>=1.7; import path may differ by version
# Pack the series into the long format the library expects.
df = pd.DataFrame({"unique_id": "load",
"ds": pd.date_range("2022-01-01", periods=H, freq="h"),
"y": load})
# MSTL handles MULTIPLE seasonality (daily=24, weekly=168) directly; the trend model
# is an AutoARIMA. Conformal prediction intervals are requested with one argument.
model = MSTL(season_length=[24, 24 * 7], trend_forecaster=AutoARIMA())
sf = StatsForecast(models=[model], freq="h",
fallback_model=AutoARIMA(),
n_jobs=-1)
# One call fits and forecasts 48 hours ahead with calibrated 90% intervals.
fc = sf.forecast(df=df, h=48, level=[90],
prediction_intervals=ConformalIntervals(n_windows=5))
print(fc[["ds", "MSTL", "MSTL-lo-90", "MSTL-hi-90"]].head(3).to_string(index=False))
ds MSTL MSTL-lo-90 MSTL-hi-90
2023-07-13 00:00:00 39.51 36.78 42.24
2023-07-13 01:00:00 38.07 35.31 40.83
2023-07-13 02:00:00 36.94 34.10 39.78
The from-scratch forecaster spelled out three things by hand: the Fourier feature matrix for daily, weekly, and yearly seasonality (about 20 lines), the ridge mean solve (about 5 lines), and the split-conformal interval (about 8 lines), roughly 50 lines with the data plumbing. Nixtla's statsforecast collapses all three to about 10 lines: MSTL performs the multiple-seasonal decomposition internally, the trend_forecaster picks and fits the trend model, and ConformalIntervals produces the calibrated band, all of it parallelized across series for the millions-of-meters case of Chapter 14. For the deep-learning version (a global neural forecaster with weather covariates and quantile heads) the sibling library neuralforecast exposes NHITS, TFT, and PatchTST behind the same interface, and Darts and GluonTS offer comparable one-call probabilistic forecasters. Reach for these in production; build from scratch once, to know what they are doing.
What the three code blocks establish together is the whole pipeline of this section in miniature. Code 35.4.1 turned the multiple seasonality of subsection one and the weather dependence into features; Code 35.4.2 produced a calibrated interval, the probabilistic object subsections three and four insisted on; the numeric example fed that interval into the reserve decision of Section 19.6; and Code 35.4.3 showed that the entire construction is a few lines of a production library. The same skeleton, swapping the weather covariate for a learned weather-model field and the linear mean for a global neural forecaster, is what a real utility runs.
Who: The day-ahead operations desk of a regional transmission operator in a summer-peaking grid with roughly 30 percent combined wind and solar penetration.
Situation: A forecast heatwave was building, with temperatures projected 8 degrees above seasonal normal for three consecutive afternoons, exactly the cooling-driven regime where the U-shaped temperature response of subsection one drives demand to record territory.
Problem: The legacy point forecaster, trained on mean squared error, systematically under-predicted extreme peaks because the squared-error optimum sits near the conditional median, not the upper tail, and the historical record held few comparable heat events to learn from.
Dilemma: Commit generation to the point forecast and risk an afternoon shortfall (and emergency purchases at punitive prices, or load shedding), or over-commit thermal plants for all three days at a large fuel and emissions cost on the chance the heat under-delivered.
Decision: The desk switched the peak-hour decision from the point forecast to the upper edge of a conformal 95 percent interval built on a multi-seasonal forecaster with the numerical-weather temperature ensemble as covariate, sizing reserve to the interval width exactly as in the numeric example above, and cross-checking the solar contribution against a downscaled GenCast-style probabilistic cloud forecast.
How: The forecaster (an MSTL plus quantile-neural ensemble of the Code 35.4.3 shape) ingested each member of the weather ensemble, producing a load distribution rather than a number, and reserve was set to cover the 95th percentile of total net load (demand minus the lower-tail renewable forecast).
Result: All three peaks landed inside the committed band; the grid held without emergency action, the reserve carried on the two milder afternoons was modest and known in advance, and the post-event review credited the calibrated upper tail, not a better point forecast, for the outcome.
Lesson: In energy the value lives in the tail, and a calibrated interval fed into an asymmetric decision (Section 19.6) beats a sharper point forecast that the operator cannot safely act on. Forecast the distribution, then dispatch to the quantile the cost structure demands.
6. Synthesis: Energy and Climate as the Book in One Application Intermediate
Energy and climate draw on more of this book at once than almost any other application, which is why we placed them here, near its end. The demand series is the multiple-seasonality decomposition of Chapter 3 and the SARIMA and Prophet models of Chapter 5, scaled to millions of series by the hierarchical and global models of Chapter 14. The renewable and weather layers are the spatio-temporal fields of Chapter 32, now realized at planetary scale by the learned weather models whose lineage runs straight through the Fourier operators, Transformers, graph networks, and diffusion ensembles of Parts III and IV. The decision layer is the probabilistic forecasting and decision-theoretic evaluation of Chapter 19 and the sequential control of Part VI. The worked forecaster tied the chain together: seasonal features, a calibrated interval, and a reserve decision sized to the tail.
The application also clarifies the book's central claim. Forecasting is not the goal; it is the input to a decision whose cost structure is asymmetric and physical, and a temporal AI system that forecasts beautifully but ignores the decision it feeds has solved the wrong problem. Energy makes that lesson concrete because the cost of ignoring it is measured in fuel, emissions, and the lights staying on. The neighboring sections extend the same lesson to other domains: Section 35.3 traced it through manufacturing and predictive maintenance, and Section 35.5 next carries it into autonomous systems and robotics, where the forecast becomes a prediction of the world the robot must act in. The full map of applications, and the parts each draws on, lives on the chapter index.
If one idea survives this section it is that in energy and climate the valuable output is the calibrated tail of a distribution, not the center. The grid is sized for the peak, the reserve is sized for the worst plausible shortfall, the extreme weather event is the one that breaks records and budgets, and the long-horizon forecast is honest only as a distribution. Every part of the book that taught uncertainty (probabilistic and conformal forecasting in Chapter 19, generative ensembles in Chapter 17, the decision-theoretic loss of Section 19.6) is what makes that tail trustworthy, and every part that taught structure (seasonality, spatio-temporal fields, hierarchy) is what makes it sharp. A sharp, calibrated tail fed into an asymmetric decision is the entire product.
Exercises
- Conceptual. A load series shows clear daily and weekly seasonality and a strong dependence on temperature. A colleague proposes a plain RNN trained on the raw load alone, with no calendar or weather features, arguing that a sufficiently deep network "will learn the seasonality itself." Give two distinct reasons, one about sample efficiency and one about the weather forecast as a covariate, why explicitly providing Fourier-seasonal features and a temperature forecast is preferable, and explain what the RNN can still usefully add on top of those features (relate your answer to the calendar-times-weather-plus-residual decomposition of subsection one).
- Implementation. Extend Code 35.4.2 to produce asymmetric prediction intervals by computing separate conformal radii for the upper and lower tails, using the signed calibration residuals and the $\alpha/2$ and $1-\alpha/2$ quantiles of the signed residual distribution. Verify on the test set that the upper and lower coverage each match their targets, and explain why an asymmetric interval is the more natural object for the heteroscedastic, summer-amplified load of Code 35.4.1 and for the reserve decision of the numeric example.
- Open-ended. A learned global weather model (GraphCast-style) produces a single deterministic field, while GenCast produces an ensemble of fields. You must forecast the next-day power output of a fleet of geographically spread wind and solar farms with calibrated intervals. Sketch a pipeline that ingests the weather model output, propagates its uncertainty through the wind power curve and solar geometry of subsection two, accounts for the spatial correlation across the fleet (Chapter 32), and emits a calibrated joint distribution of fleet output. Discuss what changes when the input is the deterministic field versus the ensemble, and where in the pipeline the spatio-temporal correlation must be modeled rather than assumed independent.