Windowing, Segmentation, and Lag Construction

"I am the same fixed length I always was, yet I never sit still. Every step I slide one tick into the future, and the past I carry forward is never quite the past I carried a moment ago."

A Sliding Window With No Fixed Address

The previous section, Section 2.4, built clean feature representations of a series: lags, rolling statistics, calendar encodings, and the rest of the engineered inputs. This section answers the next question: given those features, how do you carve the continuous record into the discrete supervised samples a model actually consumes? The answer is windowing, and it is deceptively simple to get almost right and quietly wrong. We proceed from the framing (series to samples), through the window-type taxonomy and the multi-step forecasting strategies, to the one vectorized function you will copy into every project, and finally to segmentation for classification where overlap and leakage interact in a way that has embarrassed many a published result.

1. From a Series to Supervised Samples Beginner

Let the observed series be $x_1, x_2, \ldots, x_T$, where each $x_t \in \mathbb{R}^{d}$ may be a scalar ($d = 1$) or a vector of $d$ channels. A supervised forecasting sample anchored at time $t$ pairs a lookback window of the most recent $L$ observations with a target horizon of the next $H$ observations. Writing $x_{a:b}$ for the contiguous block $(x_a, x_{a+1}, \ldots, x_b)$, the input and target are

Here $L$ is the lookback (also called the context length or window size), $H$ is the horizon (how many steps ahead you predict), and the anchor $t$ is the boundary between known past and unknown future. The single non-negotiable invariant is that the entire input lies strictly before the entire target: $\max(\text{input index}) = t < t+1 = \min(\text{target index})$. Violate it and you have built look-ahead bias directly into your dataset, the cardinal sin catalogued in Section 1.6 and policed throughout Section 2.6.

To generate many samples you slide the anchor forward by a fixed stride $S$. Starting from the first position where a full lookback exists, the valid anchors are $t \in \{L, L+S, L+2S, \ldots\}$ subject to $t + H \le T$ so that the target also fits. The number of samples produced is therefore

a formula worth internalizing because it tells you immediately how a longer horizon or a larger stride shrinks your dataset. The symbols $L$, $H$, $S$, $T$, and the index conventions used here are fixed in the unified notation table of Appendix A; we use them consistently for the rest of the book.

2. Window Types: Sliding, Tumbling, and Expanding Beginner

The stride $S$ relative to the lookback $L$ defines three classic window regimes, and the choice governs how many samples you get and how correlated they are. A sliding window uses $S < L$ (very commonly $S = 1$), so consecutive windows overlap heavily and share most of their content. A tumbling window uses $S = L$, so windows abut without overlapping and partition the series into disjoint blocks. An expanding window keeps its left edge pinned at the start and grows its right edge, so the lookback length itself increases with $t$ rather than staying fixed; this is the natural shape for cumulative statistics and for the walk-forward evaluation of Section 2.6. Figure 2.1 shows the shared mechanism: a frame gliding along the series and stamping out fixed-shape samples.

The trade-off is a direct tension between sample count and sample independence. Sliding windows with $S = 1$ maximize the number of training samples, which helps data-hungry deep models, but adjacent samples differ by a single time step and are therefore almost perfectly correlated, so the effective sample size is far smaller than $N$ suggests. Tumbling windows yield far fewer samples but those samples are much closer to independent, which makes them safer for honest variance estimates and statistical tests. The danger that unites both, and that Section 5 below treats in full, is leakage across the train-validation boundary: if a sliding window centered just before the split and another just after share overlapping timestamps, the validation set contains data the training set has already seen.

3. Direct, Recursive, and Multi-Output Forecasting Intermediate

Once $H > 1$ you must decide how the model produces multiple future steps, and three strategies dominate. The recursive (iterated) strategy trains a single one-step model and rolls it out: predict $\hat{x}_{t+1}$, feed it back as if observed, predict $\hat{x}_{t+2}$, and so on for $H$ steps. The direct strategy trains $H$ separate models, one per horizon, so model $h$ maps the lookback straight to $x_{t+h}$ with no feedback. The multi-output (joint) strategy trains one model whose output layer emits the entire horizon vector $\hat{Y}_t \in \mathbb{R}^{H \times d}$ in a single forward pass; this is what sequence-to-sequence and most modern deep forecasters do.

The strategies differ sharply in error behavior. Recursive forecasting is cheap to train (one model) but compounds error, because each predicted step becomes a noisy input to the next and small mistakes snowball across the horizon. Direct forecasting avoids that feedback loop and so resists error accumulation, but it trains and stores $H$ models and lets each horizon learn an unrelated function, ignoring the smoothness that ties neighboring horizons together. Multi-output forecasting shares representation across the whole horizon and predicts it consistently in one pass, at the cost of a heavier output head and a fixed horizon baked into the architecture. Table 2.5.1 lays the trade-offs side by side.

No strategy dominates universally. Recursive shines for short horizons on smooth series and when you need a flexible, retrainable horizon; direct wins when horizons behave very differently and compute is cheap relative to accuracy; multi-output is the default for deep learning and is what the architectures of Chapter 14 assume. The classical autoregressive roots of recursion are developed in Chapter 5, and the foundation-model heads that emit a whole horizon at once appear in Chapter 15.

4. Building the Tensors: A Reusable Windowing Function Intermediate

Theory in hand, here is the function you will actually reuse. Code 2.5.1 builds the $(N, L, d)$ input tensor and the $(N, H, d)$ target tensor from a raw array, honoring lookback $L$, horizon $H$, and stride $S$, with an explicit correctness assertion that no target index ever precedes its input. We use NumPy stride tricks (sliding_window_view) so the windowing is a memory view rather than a copy, which matters once $T$ reaches the millions.

import numpy as np
from numpy.lib.stride_tricks import sliding_window_view

def make_windows(series, lookback, horizon, stride=1):
    """Turn a (T,) or (T, d) series into supervised windows.

    Returns
    -------
    X : (N, lookback, d)   input windows  x_{t-L+1 : t}
    Y : (N, horizon, d)    target windows x_{t+1 : t+H}
    """
    x = np.asarray(series, dtype=np.float64)
    if x.ndim == 1:                       # promote scalar series to (T, 1)
        x = x[:, None]
    T, d = x.shape
    span = lookback + horizon             # total length each sample covers
    if T < span:
        raise ValueError(f"series length {T} < lookback+horizon {span}")

    # All length-`span` slices, then split each into past (L) and future (H).
    frames = sliding_window_view(x, window_shape=span, axis=0)  # (T-span+1, d, span)
    frames = frames.transpose(0, 2, 1)[::stride]               # (N, span, d), apply stride
    X = frames[:, :lookback, :]           # x_{t-L+1 : t}
    Y = frames[:, lookback:, :]           # x_{t+1   : t+H}

    # Correctness: the last input index must precede the first target index
    # for every single sample. This is the leakage invariant, asserted, not hoped.
    last_in  = np.arange(X.shape[0]) * stride + lookback - 1
    first_out = last_in + 1
    assert np.all(first_out > last_in), "target precedes input: window is leaky"
    return X, Y

The usage example in Code 2.5.2 runs the function on a short synthetic series and prints the resulting tensor shapes alongside the sample-count formula from Section 1, confirming they agree. It also shows the first input window and its target so you can read the temporal boundary directly off the numbers.

# A tiny ramp series 0,1,2,... makes the index alignment readable by eye.
series = np.arange(20.0)
L, H, S = 4, 2, 2
X, Y = make_windows(series, lookback=L, horizon=H, stride=S)

T = len(series)
N_formula = (T - L - H) // S + 1
print("X shape:", X.shape, " Y shape:", Y.shape)
print("N from formula:", N_formula, " N actual:", X.shape[0])
print("first input :", X[0, :, 0])     # x_{1:4}  -> [0 1 2 3]
print("first target:", Y[0, :, 0])     # x_{5:6}  -> [4 5]

X shape: (8, 4, 1)  Y shape: (8, 2, 1)
N from formula: 8  N actual: 8
first input : [0. 1. 2. 3.]
first target: [4. 5.]

5. Segmentation for Classification and Events Advanced

Forecasting is not the only consumer of windows. In temporal classification, human activity recognition, sleep staging, machine-fault detection, you cut the series into segments and assign each a label rather than a future. Two anchoring schemes compete. Fixed windows tile the series on a regular grid (often with a chosen overlap) and label each by majority vote of the timestamps it covers. Event-anchored windows instead center a window on each event of interest (a detected heartbeat, an alarm, a button press) and label it by that event's class, which keeps the phenomenon centered but produces irregularly spaced, possibly overlapping segments.

Label alignment is the subtle part. When a fixed window spans a transition between two activities, majority vote can mislabel it, so practitioners either shrink the window, require a purity threshold (discard windows that are not at least, say, eighty percent one class), or label by the final timestamp when the task is causal. None of this matters, however, next to the overlap-aware splitting rule, because overlapping classification windows are the single most common source of inflated accuracy in the applied temporal-ML literature.

The safe segmentation recipe is therefore mechanical. First split the raw series (or the cohort of subjects) into disjoint train, validation, and test partitions by time or by identity. Then window each partition independently. Then, at every internal boundary, purge a gap of at least one full window length so no surviving window overlaps across the split. Group-aware cross-validators such as scikit-learn's GroupKFold and the purged, embargoed schemes of Section 2.6 automate exactly this, and the same overlap-leakage trap, viewed from the evaluation side, is introduced in Section 1.6 and developed as the central concern of Section 2.6.