Backpropagation Through Time

"They sent me backward from the loss at the final step, and I have been crawling ever since: through step one thousand, step nine hundred, step eight hundred, multiplying by one Jacobian after another. Somewhere around step forty I stopped feeling anything at all. By the time I reach the first weight I am a rounding error with ambitions."

A Gradient Crawling Backward Through a Thousand Timesteps

Section 9.2 settled the task taxonomy and the padded, masked recurrent encoder: what shape the model must emit and how variable-length sequences are batched. The recurrent cell whose forward pass we now differentiate is the carried-state loop $\mathbf{h}_t = f(\mathbf{h}_{t-1}, \mathbf{x}_t)$ that Section 7.6 distilled from the classical filters; here we fix it to the canonical Elman form $\mathbf{h}_t = \tanh(\mathbf{W}\mathbf{h}_{t-1} + \mathbf{U}\mathbf{x}_t)$ and answer the one question that makes the cell learn: given a loss measured at the outputs, how does the gradient reach the shared weights buried inside the loop? This section answers that question completely. The recurrence at the center of everything is the same skeleton we met as the closing bridge of Part II, where Section 7.6 showed that the Kalman filter, the HMM forward pass, and the RNN are one loop, $\mathbf{h}_t = f(\mathbf{h}_{t-1}, \mathbf{x}_t)$, $\hat{\mathbf{y}}_t = g(\mathbf{h}_t)$, with different choices of $f$ and $g$. Part II fixed $f$ and $g$ by hand from a model. Part III learns them by gradient descent, and the gradient is computed by the algorithm we now build. We use the unified notation of Appendix A throughout: $\mathbf{x}_t$ the input, $\mathbf{h}_t$ the hidden state, $\hat{\mathbf{y}}_t$ the prediction, $\mathcal{L}$ the total loss.

Why does this deserve a section of its own rather than a passing remark that "you just run backpropagation"? Because the weight sharing that defines a recurrence turns an otherwise routine gradient computation into the single most consequential algorithm in temporal deep learning. The same shared weight that lets an RNN process a sequence of any length with a fixed parameter count is what forces its gradient to be a sum across all those lengths, and that sum, threaded through a product of Jacobians, is the origin of both the expressive power and the training difficulty of every recurrent model. Get BPTT right and the recurrent cell learns; misunderstand it and you will write a training loop that silently leaks memory, computes a biased gradient you did not intend, or fails to learn a dependency the architecture was perfectly capable of representing. The algorithm is short, but its consequences run through the rest of Part III.

The four concrete competencies this section installs are these: to unroll a recurrence into its computation graph and see the weight sharing; to write the loss gradient with respect to a shared recurrent weight as a sum over timesteps of a product of Jacobians, and to derive that sum for a simple cell; to explain why truncated BPTT bounds compute and memory at the price of a bias, and how a library implements truncation with a single detach; and to verify a hand-written BPTT against autograd to floating-point agreement. These are the load-bearing skills for every recurrent architecture in Chapter 10 and beyond.

1. The Recurrent Computation Graph: Unrolling the Recurrence Beginner

A recurrence is defined by a single equation that refers to itself: the state at time $t$ depends on the state at time $t-1$ and the current input. Written once, it looks like a loop with no depth at all. The key conceptual move, the move that makes gradient computation possible, is to unroll that loop in time: write out one copy of the cell for each timestep, feed the hidden state of each copy into the next, and you obtain an ordinary feedforward graph that is exactly as deep as the sequence is long. The recurrence has not changed; we have only drawn it differently, and the new drawing is one to which ordinary backpropagation applies.

The reason this reframing is not merely cosmetic is that backpropagation is defined on directed acyclic computation graphs, and a recurrence, taken literally, is cyclic: $\mathbf{h}_t$ depends on $\mathbf{h}_{t-1}$ which depends on $\mathbf{h}_{t-2}$ and so on, a self-reference with no obvious place to attach the chain rule. Unrolling is precisely the act of cutting that cycle open along the time axis. Once each timestep gets its own node, the cycle becomes a chain, the chain is acyclic, and the standard reverse-mode automatic differentiation that powers every deep-learning framework applies without modification. So the entire conceptual content of "backpropagation through time" is in those three words "through time": we are doing nothing but ordinary backpropagation, along the one extra axis the recurrence introduced. Everything that follows is the consequence of that single reframing.

Concretely, fix the cell to the canonical Elman form that opens Chapter 10, with a single shared recurrent weight $\mathbf{W}$, input weight $\mathbf{U}$, and readout weight $\mathbf{V}$:

Here $\mathbf{a}_t$ is the pre-activation, $\mathbf{h}_t = \tanh(\mathbf{a}_t)$ the activated hidden state, and $\hat{\mathbf{y}}_t$ the readout. Unrolling over $t = 1, \dots, T$ produces the chain

with a readout $\hat{\mathbf{y}}_t = \mathbf{V}\mathbf{h}_t$ branching off each state and a per-step loss $\ell_t(\hat{\mathbf{y}}_t, y_t)$ branching off each readout. The total loss is the sum over time, $\mathcal{L} = \sum_{t=1}^{T} \ell_t$. The single fact that distinguishes this from an ordinary deep network is printed on every arrow: the same $\mathbf{W}$, $\mathbf{U}$, $\mathbf{V}$ appear at every timestep. The unrolled net is a $T$-layer feedforward net with tied weights, and that tying is the entire source of what makes BPTT interesting. Figure 9.3.1 draws the unrolled graph with the shared weights labeled on each arrow.

The unrolled view immediately explains two practical facts. First, the graph is as deep as the sequence is long, so a model trained on sequences of length one thousand is, for the purpose of backpropagation, a one-thousand-layer network, which is why recurrent training inherits the gradient pathologies of very deep networks (the subject of Section 9.4). Second, every intermediate $\mathbf{h}_t$ produced on the forward pass must be kept in memory until the backward pass consumes it, because the gradient of a $\tanh$ needs the value it was applied to. That storage requirement, linear in $T$, is the central cost we manage in subsection four.

It is worth dwelling on the difference between the unrolled picture and a genuinely deep feedforward network, because the difference is the single source of everything subtle about BPTT. In a plain $T$-layer network each layer has its own weight matrix, so a parameter appears exactly once and its gradient is a single chain-rule term. In the unrolled recurrence the one matrix $\mathbf{W}$ is stamped onto every layer, so it appears $T$ times and its gradient is a sum of $T$ chain-rule terms, one per appearance. The unrolling is a conceptual device, not a change to the model: at no point do we actually instantiate $T$ separate matrices in memory: there is one $\mathbf{W}$, and the gradient bookkeeping simply adds up the contribution it makes at each timestep. Holding "one matrix, many appearances, summed gradient" firmly in mind is what keeps the derivation of subsection two from feeling like sleight of hand.

2. Backpropagation Through Time: The Chain Rule Across Steps Intermediate

We now derive the gradient of the total loss with respect to the shared recurrent weight $\mathbf{W}$. The derivation has exactly two ingredients, and keeping them separate is what makes the algebra clean rather than intimidating:

• the multi-use sum rule: a parameter that appears in several places in a graph receives a gradient contribution from each appearance, and the total is their sum;
• the chain rule along the state chain: influence from one timestep reaches a later one by passing through every intermediate hidden state, multiplying one hidden-to-hidden Jacobian at each.

The sum rule is the price of weight sharing and the chain rule is the price of depth. The first produces a sum over timesteps; the second produces a product of Jacobians inside each summand. The entire derivation below is just these two facts applied carefully, and every difficulty of recurrent training traces to one or the other.

Start from the total loss $\mathcal{L} = \sum_{t=1}^{T} \ell_t$. Because $\mathbf{W}$ feeds into the pre-activation $\mathbf{a}_\tau$ at every step $\tau$, and each $\mathbf{a}_\tau$ influences the loss through its own state and every later state, the total derivative is a sum over the steps where $\mathbf{W}$ is used:

Take one term $\partial \ell_t / \partial \mathbf{W}$. The loss at step $t$ sees $\mathbf{W}$ through the hidden state $\mathbf{h}_t$, but $\mathbf{h}_t$ was built from $\mathbf{h}_{t-1}$, which used $\mathbf{W}$, which was built from $\mathbf{h}_{t-2}$, which used $\mathbf{W}$, and so on back to the start. So $\mathbf{W}$ reaches $\ell_t$ through every step $k \le t$. Applying the chain rule and the multi-use sum rule together,

where $\partial^{+}\mathbf{h}_k / \partial \mathbf{W}$ is the immediate (local) derivative of $\mathbf{h}_k$ with respect to $\mathbf{W}$, treating $\mathbf{h}_{k-1}$ as a constant, and the bracketed product is the chain of hidden-to-hidden Jacobians carrying influence from step $k$ forward to step $t$. Summing over $t$ and reindexing gives the full shared-weight gradient. The decisive object is the product of Jacobians: define the per-step hidden-to-hidden Jacobian

which follows because $\mathbf{h}_j = \tanh(\mathbf{W}\mathbf{h}_{j-1} + \mathbf{U}\mathbf{x}_j)$ and $\tanh'(a) = 1 - \tanh^2(a)$, so the diagonal matrix holds the elementwise activation derivatives and $\mathbf{W}$ is the linear map inside. The product $\prod_{j=k+1}^{t} \mathbf{J}_j$ is what propagates a gradient from step $t$ back to step $k$, and it is the same object whose magnitude decides, in Section 9.4, whether the gradient vanishes (Jacobian norms below one, product shrinks geometrically) or explodes (norms above one, product grows geometrically).

In practice we do not form these products explicitly. We accumulate the backward pass with a single recurrence in the adjoint $\boldsymbol{\delta}_t \equiv \partial \mathcal{L} / \partial \mathbf{h}_t$ (the backprop error signal, the gradient of the total loss with respect to the hidden state at step $t$, the same quantity called delta in feedforward backprop). Because $\mathbf{h}_t$ influences the loss both through its own readout and through the next state $\mathbf{h}_{t+1}$, the adjoint obeys the backward recurrence

initialized at the last step with $\boldsymbol{\delta}_T = \mathbf{V}^{\top}\,\partial \ell_T / \partial \hat{\mathbf{y}}_T$ (no future term). Once every $\boldsymbol{\delta}_t$ is known, the shared-weight gradients are simple per-step sums, each picking up the local contribution and adding it across time:

This adjoint recurrence is exactly what the numpy implementation in subsection five computes, and it is exactly what PyTorch autograd computes for you when you call .backward() on the unrolled graph. The two will agree to floating-point precision because they are the same arithmetic. Notice the structure that recurs everywhere in this section: the forward pass walks $t = 1 \to T$ accumulating states; the backward pass walks $t = T \to 1$ accumulating adjoints; the weight gradient is a sum over the whole walk.

The adjoint recurrence is worth reading as a small economy of its own. At each step, walking backward, the adjoint $\boldsymbol{\delta}_t$ receives two streams of gradient and merges them: a fresh, local stream from the loss measured at that very step, $\mathbf{V}^\top\,\partial\ell_t/\partial\hat{\mathbf{y}}_t$, and an inherited stream from the future, $\mathbf{J}_{t+1}^\top\boldsymbol{\delta}_{t+1}$, carrying everything later steps want to say about $\mathbf{h}_t$. The merge is a plain sum because $\mathbf{h}_t$ influences the loss along both routes and the chain rule adds the routes. The inherited stream is where the Jacobian product hides: every backward step multiplies the running adjoint by one more $\mathbf{J}^\top$, so a gradient travelling from step $t$ back to step $k$ has been multiplied by $t - k$ Jacobians by the time it arrives. This is the same product that appeared explicitly in the per-term formula above, now assembled incrementally rather than written out, and it is the more efficient of the two equivalent views because it never forms the product as a standalone matrix.

3. Truncated BPTT: Bounding the Backward Window Intermediate

Full BPTT, the algorithm of subsection two, backpropagates from the loss at the last step all the way to the first, multiplying one Jacobian per step. For a sequence of length $T$ this costs $O(T)$ time and, more painfully, $O(T)$ memory, because every hidden state on the forward pass must be retained for the backward pass.

For the long sequences that motivate recurrent models in the first place this is a real wall, not a theoretical inconvenience. A year of hourly sensor readings is roughly nine thousand steps; a single document is thousands of tokens; a high-frequency financial tape can be hundreds of thousands of ticks. Unrolling any of these to its full length and holding every activation is often simply impossible on available hardware. And there is a second, more insidious problem layered on top of the memory one: the very long Jacobian product is exactly where the vanishing and exploding gradients of Section 9.4 bite hardest, so the gradient contributions from the distant past are usually either negligible (vanished to nothing) or numerically unusable (exploded to overflow) anyway. We are therefore paying the full $O(T)$ memory cost to compute long-range gradient terms that, for a plain RNN, frequently carry no usable signal. Truncation refuses that bad bargain.

Truncated backpropagation through time (TBPTT) makes the obvious trade. Instead of backpropagating across the whole sequence, it processes the sequence in chunks of length $k$ and backpropagates only within each chunk (or only $k$ steps back from each loss). The forward pass still runs across the full sequence so the hidden state carries information forward without limit; only the backward pass is windowed. Mechanically, at each chunk boundary we keep the hidden state's value (so memory persists forward) but cut its gradient connection to earlier steps (so the backward pass stops). In a framework this cut is a single operation: h = h.detach() in PyTorch, which returns a tensor with the same value but no history, so autograd treats it as a constant leaf.

The cost is bounded and the bias is explicit. With window $k$, time and memory per update are $O(k)$ rather than $O(T)$, independent of how long the full sequence is. The price is that gradients are computed as if dependencies longer than $k$ steps did not exist: any genuine influence of $\mathbf{x}_{t-k-1}$ on the loss at $t$ is simply dropped, so the truncated gradient is a biased estimate of the true full-sequence gradient. The bias is small when the true long-range gradient is small (which, thanks to vanishing gradients, it often is for plain RNNs), and it is harmful precisely when the task genuinely needs long-range credit assignment, in which case truncation can prevent the model from ever learning the dependency. The window length $k$ is therefore a direct knob on the bias-versus-cost trade: large $k$ approaches full BPTT (low bias, high cost); small $k$ is cheap and stable but blind beyond its horizon.

There is a clean way to picture what truncation actually deletes from the gradient. Recall from subsection two that the exact weight gradient is a double sum, $\sum_t \sum_{k \le t} (\cdots)$, over all pairs of a loss-bearing step $t$ and an earlier weight-use step $k$. Full BPTT includes every pair with $k \le t$. Truncation with window $k$ keeps only the pairs whose separation $t - k$ is at most $k$ and discards the rest, so it is a triangular mask on the contribution matrix: the near-diagonal short-range terms survive, the far-from-diagonal long-range terms vanish. Whether that deletion matters is entirely a question of how much gradient mass lived in the discarded long-range terms, which by the Jacobian-product argument is small exactly when the per-step Jacobian norms are below one. This is why truncation and the vanishing-gradient phenomenon of Section 9.4 are two sides of one coin: the very decay that makes a plain RNN struggle to learn long dependencies is also what makes truncating its gradient nearly lossless.

A subtle and important variant is the two-parameter scheme TBPTT$(k_1, k_2)$: run the forward pass $k_1$ steps, then backpropagate $k_2$ steps (with $k_2 \ge k_1$), and slide. Setting $k_1 = k_2 = k$ recovers the simple chunked version above, and it is what most framework training loops implement when they detach the hidden state between minibatches of a long sequence. The practical default in language modeling and long-horizon forecasting is a window of tens to a couple hundred steps, chosen so the relevant dependencies fit inside the window while the cost stays affordable. We make truncation visibly change the gradient in the code of subsection five.

4. Memory and Compute: The Cost of Unrolling Advanced

The dominant resource in recurrent training is not arithmetic but memory, and understanding why explains both why truncation helps and why a second technique, gradient checkpointing, exists. The backward pass for a $\tanh$ cell needs the activation $\mathbf{h}_t$ to compute the local derivative $1 - \mathbf{h}_t^2$. So the naive forward pass must store every $\mathbf{h}_t$ for $t = 1, \dots, T$ until the backward pass consumes it in reverse order. For a hidden width $d$, a batch of $B$ sequences, and length $T$, the activation memory is $O(B\,d\,T)$, and for long sequences this term, not the model parameters, is what fills the GPU.

Gradient checkpointing trades compute for memory to break this linear growth. Instead of storing every activation, store only a sparse set of checkpoints (say every $\sqrt{T}$ steps); during the backward pass, recompute the missing activations on the fly by re-running short forward segments from the nearest stored checkpoint. The classic result is that storing $O(\sqrt{T})$ checkpoints and recomputing the rest reduces activation memory from $O(T)$ to $O(\sqrt{T})$ at the cost of one extra forward pass overall, a roughly $1.3\times$ to $2\times$ slowdown. For the very long sequences of Chapter 13 and the long-context Transformers of Chapter 12, checkpointing is what makes training fit in memory at all. It is exposed as torch.utils.checkpoint and is orthogonal to truncation: checkpointing computes the exact gradient with less memory, whereas truncation computes a biased gradient with less memory and less compute.

The asymmetry between the two memory-saving techniques is worth stating plainly because practitioners routinely confuse them. Truncation and checkpointing both reduce the memory wall of the unrolled graph, but they pay for it in different currencies and they produce different gradients. Truncation pays with bias: it computes a gradient that is genuinely different from, and generally worse than, the full-sequence gradient, because it has thrown information away. Checkpointing pays with recompute: it computes the exact full-sequence gradient, identical to naive BPTT to the last bit, and the only cost is running parts of the forward pass twice. The decision rule follows directly: if your sequences are long because the task has long dependencies you must learn, reach for checkpointing (or a longer truncation window plus checkpointing) so the gradient stays honest; if the dependencies you care about are genuinely short and the long sequence is merely a streaming convenience, truncation is the cheaper and entirely appropriate choice. Reaching for truncation when you needed exactness is the quiet failure that produces a model that trains smoothly and forecasts badly.

Two batching considerations round out the compute picture. First, sequences in a batch usually differ in length, so they are padded to a common length and a mask prevents the padded steps from contributing to the loss or the gradient; without the mask the padding silently corrupts the weight-gradient sums of subsection two. Second, the time loop is inherently sequential, step $t$ cannot start until step $t-1$ finishes, so a recurrent layer cannot parallelize across time the way a convolution or an attention layer can. This sequential dependency is the efficiency weakness that the temporal-convolutional networks of Chapter 11, the attention models of Chapter 12, and the parallel-scan state-space models of Chapter 13 were each designed to overcome, and it is the practical reason the field moved beyond the vanilla RNN even though the recurrence is conceptually elegant.

5. Worked Example: BPTT by Hand, Then by Autograd Advanced

We now make every equation of subsection two executable. The plan is the cleanest possible demonstration of correctness: implement the forward pass and the manual backward pass of a tiny RNN entirely in numpy, with the weight gradients accumulated step by step exactly as the adjoint recurrence prescribes; then build the identical cell in PyTorch, let autograd compute the gradients; then assert the two sets of gradients agree to floating-point precision with np.allclose. If they match, the hand derivation is verified against an independent implementation, which is the strongest evidence a derivation can have. Code 9.3.1 is the from-scratch forward and manual backward.

import numpy as np

rng = np.random.default_rng(0)
d_in, d_h, d_out, T = 3, 4, 2, 6        # input, hidden, output widths; seq length

# Shared weights (the SAME W, U, V used at every timestep).
W = rng.normal(0, 0.5, size=(d_h, d_h))   # recurrent weight h_{t-1} -> h_t
U = rng.normal(0, 0.5, size=(d_h, d_in))  # input weight     x_t     -> h_t
V = rng.normal(0, 0.5, size=(d_out, d_h)) # readout weight   h_t     -> y_hat_t

xs = rng.normal(0, 1.0, size=(T, d_in))   # input sequence
ys = rng.normal(0, 1.0, size=(T, d_out))  # targets (squared-error loss)

def forward(W, U, V, xs, h0):
    """Run the recurrence forward, caching every state for the backward pass."""
    hs = [h0]                              # hs[t] is h_t; hs[0] = h0
    preds = []
    for t in range(T):
        a = W @ hs[-1] + U @ xs[t]         # pre-activation a_t
        h = np.tanh(a)                     # h_t = tanh(a_t)
        hs.append(h)
        preds.append(V @ h)                # y_hat_t = V h_t
    return hs, np.array(preds)

h0 = np.zeros(d_h)
hs, preds = forward(W, U, V, xs, h0)
loss = 0.5 * np.sum((preds - ys) ** 2)     # total loss L = sum_t 0.5||y_hat_t - y_t||^2

def bptt(W, U, V, xs, ys, hs, preds):
    """Manual BPTT: the adjoint recurrence and summed weight gradients of subsection 2."""
    dW = np.zeros_like(W); dU = np.zeros_like(U); dV = np.zeros_like(V)
    dh_next = np.zeros(d_h)                 # delta carried from the future, starts at 0
    for t in reversed(range(T)):            # walk t = T-1 ... 0
        dy = preds[t] - ys[t]              # d ell_t / d y_hat_t  (squared error)
        dV += np.outer(dy, hs[t + 1])      # dL/dV picks up h_t = hs[t+1]
        dh = V.T @ dy + dh_next            # delta_t = local readout term + from the future
        da = (1 - hs[t + 1] ** 2) * dh     # through tanh': (1 - h_t^2) elementwise
        dW += np.outer(da, hs[t])          # dL/dW picks up h_{t-1} = hs[t]
        dU += np.outer(da, xs[t])          # dL/dU picks up x_t
        dh_next = W.T @ da                 # propagate adjoint to step t-1: J_t^T delta_t
    return dW, dU, dV

dW, dU, dV = bptt(W, U, V, xs, ys, hs, preds)
print("manual loss = %.6f" % loss)
print("manual dW[0,0] = %.6f" % dW[0, 0])

manual loss = 5.341792
manual dW[0,0] = -0.118734

Now the library pair. Code 9.3.2 builds the same cell with the same weights as PyTorch tensors, runs the identical recurrence, and calls loss.backward(). Autograd constructs the unrolled graph behind the scenes and runs exactly the adjoint recurrence we wrote by hand. We then compare every gradient with np.allclose, the gradient-check that certifies the hand BPTT.

import torch

# Same weights as Code 9.3.1, now as autograd-tracked tensors.
Wt = torch.tensor(W, requires_grad=True)
Ut = torch.tensor(U, requires_grad=True)
Vt = torch.tensor(V, requires_grad=True)
xt = torch.tensor(xs); yt = torch.tensor(ys)

h = torch.zeros(d_h, dtype=torch.float64)
loss_t = 0.0
for t in range(T):                          # the SAME recurrence, now traced by autograd
    h = torch.tanh(Wt @ h + Ut @ xt[t])     # h_t = tanh(W h_{t-1} + U x_t)
    pred = Vt @ h                           # y_hat_t = V h_t
    loss_t = loss_t + 0.5 * ((pred - yt[t]) ** 2).sum()

loss_t.backward()                           # autograd runs BPTT for us, in one call

# Gradient check: hand-written BPTT vs autograd, to floating point.
print("loss match :", np.allclose(loss, loss_t.item()))
print("dW match   :", np.allclose(dW, Wt.grad.numpy()))
print("dU match   :", np.allclose(dU, Ut.grad.numpy()))
print("dV match   :", np.allclose(dV, Vt.grad.numpy()))

loss match : True
dW match   : True
dU match   : True
dV match   : True

Finally we make truncation visible. Code 9.3.3 computes the gradient three ways on the same sequence: full BPTT, then truncated BPTT with a short window, then a shorter window still, and prints how the recurrent-weight gradient drifts away from the exact full-BPTT value as the window shrinks. This is the bias of subsection three made concrete, and the mechanism is exactly the one-line detach discussed there.

def tbptt_grad(k):
    """Gradient of W under truncated BPTT with backward window k (in PyTorch)."""
    Wt = torch.tensor(W, requires_grad=True)
    Ut = torch.tensor(U, requires_grad=True)
    Vt = torch.tensor(V, requires_grad=True)
    h = torch.zeros(d_h, dtype=torch.float64)
    total = 0.0
    for t in range(T):
        if t % k == 0 and t > 0:
            h = h.detach()                  # CUT the gradient here: forget the past
        h = torch.tanh(Wt @ h + Ut @ xt[t]) # state VALUE still flows forward
        total = total + 0.5 * ((Vt @ h - yt[t]) ** 2).sum()
    total.backward()
    return Wt.grad.numpy()

g_full = tbptt_grad(T)                       # window = T is full BPTT (no detach fires)
g_k3   = tbptt_grad(3)                        # detach every 3 steps
g_k2   = tbptt_grad(2)                        # detach every 2 steps
print("full BPTT     dW[0,0] = %.6f" % g_full[0, 0])
print("trunc k=3     dW[0,0] = %.6f  (drift %.4f)" % (g_k3[0, 0], abs(g_k3[0,0]-g_full[0,0])))
print("trunc k=2     dW[0,0] = %.6f  (drift %.4f)" % (g_k2[0, 0], abs(g_k2[0,0]-g_full[0,0])))

full BPTT     dW[0,0] = -0.118734
trunc k=3     dW[0,0] = -0.121509  (drift 0.0028)
trunc k=2     dW[0,0] = -0.130118  (drift 0.0114)

Step back and read what the three code blocks establish together. Code 9.3.1 implemented the adjoint recurrence and summed weight gradients of subsection two by hand. Code 9.3.2 reproduced every one of those numbers with a single autograd call, certifying the derivation. Code 9.3.3 turned full BPTT into truncated BPTT with one detach per chunk boundary and watched the gradient acquire exactly the bias subsection three predicted. The pedagogical payoff is that BPTT is not a black box: you can compute it by hand, a framework computes the same thing for you, and truncation is a single, visible, controllable modification of that computation.

import torch.nn as nn

# The cell, the unroll, and BPTT-ready autograd in three lines.
rnn = nn.RNN(input_size=d_in, hidden_size=d_h, batch_first=True)  # carries state, ties weights
out, h_final = rnn(torch.tensor(xs).unsqueeze(0))                 # (1, T, d_in) -> per-step states
loss = out.pow(2).mean(); loss.backward()                        # one call runs full BPTT
# Truncation is still just: detach the carried state between chunks before re-feeding it.

We have computed the gradient that trains every recurrent model in this book and verified it against autograd to the last bit. But the derivation surfaced an object we set aside: the product of per-step Jacobians $\prod_{j=k+1}^{t} \mathbf{J}_j$, with $\mathbf{J}_j = \operatorname{diag}(1 - \mathbf{h}_j^2)\,\mathbf{W}$. In the first numeric example a single factor was already $0.18$, well below one; chain hundreds of such factors across a long sequence and the gradient from the distant past is annihilated, or, if the factors exceed one, it blows up. That product is the hinge on which long-range learning turns, and it is exactly the subject of the next section. Section 9.4 studies when this Jacobian product vanishes or explodes, what that does to a network's ability to learn long-term dependencies, and the architectural and algorithmic fixes (gradient clipping, careful initialization, and the gated cells that lead into Chapter 10) that keep the gradient alive across time.