Privacy and Federated Temporal Learning

"I was trained on a million calendars and I swore an oath: I would learn that people commute and sleep and pay rent, but never that you left for the clinic at 7:14 last Tuesday. Then someone queried me carefully, and I realized I had been keeping your Tuesday in a back pocket the whole time. Differential privacy is the noise I add so that even I cannot find it again."

A Model That Promised Never to Memorize Your Tuesday

In Section 33.3 we hardened temporal models against adversarial and distributional perturbations, asking whether a model behaves sensibly when its inputs are attacked. This section asks the opposite question: whether a model protects the people whose data formed its training set. The two are the trustworthiness coin's two faces, robustness guarding the model from the world and privacy guarding the world from the model. The data we worry about is the same data that has run through this entire book. The mobility traces of Section 32.1, the irregularly sampled clinical vitals that thread from Chapter 2 through Chapter 13, and the high-frequency financial tapes of Chapter 5 are exactly the streams that are most valuable to model and most dangerous to expose. We use the unified notation of Appendix A: $\mathbf{x}_{1:T}$ a series, $\mathcal{D}$ a dataset of series, $\mathcal{M}$ a randomized training mechanism, and $\varepsilon, \delta$ the privacy parameters defined below.

Why does temporal data warrant its own privacy treatment rather than the generic tabular machinery? Because time multiplies identifiability in two directions at once. Across the sequence, the pattern of values is identifying even when no single value is: four hourly locations pin most people uniquely, as the mobility literature established. And within the learning algorithm, the correlation between adjacent time steps means a single person contributes many statistically dependent records, so a privacy budget computed as if those records were independent is optimistic, sometimes badly so. The first fact is why we cannot rely on aggregation; the second is why we cannot reuse tabular DP accounting unchanged. Both are handled below.

The competencies this section installs are four. To explain, with a concrete re-identification example, why temporal data resists anonymization. To state the $(\varepsilon, \delta)$ differential-privacy guarantee precisely and to implement DP-SGD, clipping and noising per-example gradients while tracking a privacy budget. To run a federated round with FedAvg and to name what secure aggregation and non-IID temporal data add to the picture. And to position synthetic data and membership inference as the audit around all of it. Each competency is paired with runnable code and a from-scratch-then-library demonstration.

1. Why Temporal Data Is Especially Privacy-Sensitive Beginner

Start from the uncomfortable empirical fact that anchors this whole section: a human trajectory is almost a fingerprint. The landmark study of de Montjoye and colleagues on fifteen months of mobility for one and a half million people found that just four spatio-temporal points, four times you were at a place at an hour, uniquely identify ninety-five percent of individuals, and that coarsening the spatial and temporal resolution barely helps because human movement is so individual. This is the mobility thread we opened in Section 32.1 returning with a warning attached: the very structure that makes a trajectory predictable, its regularity and individuality, is what makes it identifying. The same holds for the clinical running dataset of this book (the irregular vitals reachable from the contents): a short window of heart rate, blood pressure, and their timing is a biometric signature, and credit-card tapes are no better, four transactions reidentify ninety percent of shoppers in the companion financial study.

The practical consequence is that the standard privacy reflex, "aggregate it and it is safe", fails on time series. Aggregation removes identity from a single snapshot but not from a pattern across snapshots. A daily total hides which minute you were home; a sequence of daily totals reveals your weekday-versus-weekend rhythm, your vacations, the week you were ill. Time turns a set of weak signals into a strong one because the joint distribution over many steps is far more specific than any marginal. Releasing aggregates also invites differencing attacks: if yesterday's and today's cumulative counts differ by one and you know one person joined, you have isolated that person's contribution. Privacy for temporal data must therefore be a property of the release mechanism, not a property of any one number it emits, which is exactly the shift differential privacy formalizes.

A second time-specific subtlety determines how we will account for privacy in subsection two. When we ask "how much can one person change the model?", the answer depends on how much of the data one person owns. In a tabular dataset each person is typically one row, so bounding the influence of one row bounds the influence of one person. In a temporal dataset one person may own an entire long sequence, dozens or thousands of correlated steps, and if our algorithm treats each step as an independent record, it will under-count that person's footprint and over-state the privacy it provides. The fix, user-level (or sequence-level) privacy rather than record-level, runs through everything below: we will clip and account at the granularity of the person, not the step.

2. Differential Privacy and DP-SGD for Temporal Models Intermediate

Differential privacy gives the formal guarantee that subsection one argued we need: a promise about the mechanism, robust to any side information an attacker holds. A randomized mechanism $\mathcal{M}$ is $(\varepsilon, \delta)$-differentially private if for every pair of datasets $\mathcal{D}, \mathcal{D}'$ that differ by the addition or removal of one person's data (adjacent datasets) and for every set of outcomes $S$,

Read it as a likelihood-ratio bound: whatever the mechanism outputs, that output was almost as likely to occur with your data absent as present, so an observer learns almost nothing specific to you. The parameter $\varepsilon$ is the privacy loss (smaller is stronger; $\varepsilon \approx 1$ is strong, $\varepsilon \approx 8$ is weak but common), and $\delta$ is a small probability (typically $\delta \ll 1/N$ for $N$ people) that the clean $e^\varepsilon$ bound fails. The definition's two superpowers are composition, the $\varepsilon$ of a sequence of mechanisms adds (and modern accountants do far better than naive addition), and post-processing immunity, anything you compute from a private output stays private. Both are what let us privatize an iterative training loop.

The workhorse mechanism adds Gaussian noise calibrated to a function's sensitivity, the most one person can change it. For a vector-valued query $f$ with $L_2$-sensitivity $\Delta_2 f = \max_{\mathcal{D} \sim \mathcal{D}'} \lVert f(\mathcal{D}) - f(\mathcal{D}') \rVert_2$, the Gaussian mechanism $f(\mathcal{D}) + \mathcal{N}(0, \sigma^2 \mathbf{I})$ satisfies $(\varepsilon, \delta)$-DP when the noise multiplier obeys

This single inequality is the engine of DP-SGD (Abadi et al., 2016), which makes stochastic gradient descent private by controlling the sensitivity of each update and then noising it. Two operations per step do the work. First, per-example gradient clipping: compute the gradient $\mathbf{g}_i$ for each example (or, for temporal data, each person's sequence) and rescale it to norm at most a clipping bound $C$, $\bar{\mathbf{g}}_i = \mathbf{g}_i / \max(1, \lVert \mathbf{g}_i \rVert_2 / C)$, which forces the per-person sensitivity to be exactly $C$. Second, noise addition: sum the clipped gradients, add Gaussian noise of standard deviation $\sigma C$, and average,

The clipping guarantees a known sensitivity regardless of the data; the noise hides any one clipped gradient inside it; and a privacy accountant (the moments accountant, or today's tighter Renyi-DP and PRV accountants) composes the per-step costs over all training iterations into a single $(\varepsilon, \delta)$ for the finished model. The privacy-utility trade-off is now visible as three knobs: larger $C$ admits bigger gradients but demands more noise, larger $\sigma$ buys smaller $\varepsilon$ at the cost of noisier updates, and more steps spend more budget. Strong privacy is simply noisier, slower training, and the empirical accuracy gap it opens is the cost we measure in subsection five.

Now the time-specific crack, the one tabular treatments never mention. DP-SGD's accounting assumes that the unit whose presence we hide, the adjacency in the definition, matches the unit we clip. If we naively treat each time step (or each fixed-length window) as a separate training example and clip per window, we are providing record-level privacy: we hide one window, not one person. But a person owns many correlated windows, so hiding one of their windows leaves their footprint in all the others, and the real protection for the person is far weaker than the reported $\varepsilon$. Correlated time steps thus weaken naive privacy accounting in a precise sense: the effective number of independent contributions per person is smaller than the record count, and event-level (per-step) DP does not compose into the user-level DP we actually want. The remedy is to make the clipping unit the whole person-sequence, summing each person's per-step gradients into one vector and clipping that to $C$ before noising. Then the sensitivity bound matches person-level adjacency and the accountant's $\varepsilon$ means what we want it to mean. For streaming releases where the same person reappears over time, the stronger frameworks are user-level DP and DP under continual observation (the binary-tree mechanism), which we flag as the frontier below.

3. Federated Learning: Training Without Centralizing the Series Intermediate

Differential privacy bounds what a released model reveals; it says nothing about where the raw data sits during training. Federated learning attacks that second axis: it trains a shared model across many devices or institutions without ever pooling their raw series in one place. The data stays on the phone, in the hospital, behind the bank's firewall, which matters for temporal data both because the series are sensitive (subsection one) and because regulation and bandwidth often forbid centralizing them at all. The canonical algorithm is federated averaging (FedAvg, McMahan et al., 2017). A central server holds the global model parameters $\boldsymbol{\theta}$; each round it sends $\boldsymbol{\theta}$ to a sample of clients, each client $k$ runs a few epochs of local SGD on its own data to produce $\boldsymbol{\theta}_k$, and the server averages the returned models, weighting by each client's number of examples $n_k$:

Only model weights, never raw vitals or transactions, cross the network, and the local computation is ordinary temporal-model training of the kind built throughout Part III. Communication, not computation, is the scarce resource, which is why clients run multiple local steps per round rather than sending a gradient per step.

Two problems separate the textbook formula from a deployable system, and both are sharper for temporal data. The first is that the model updates themselves leak: a single client's $\boldsymbol{\theta}_k$, or even its difference from $\boldsymbol{\theta}$, can be inverted to reconstruct training sequences (gradient-inversion attacks recover surprisingly faithful inputs). Secure aggregation closes this hole cryptographically: clients mask their updates with pairwise-cancelling random vectors so that the server can compute the sum $\sum_k \boldsymbol{\theta}_k$ but cannot read any individual $\boldsymbol{\theta}_k$, the masks cancelling only in aggregate. Pair secure aggregation with DP noise added to the sum and you get the strong combination deployed in practice: the server sees only a differentially private average it cannot decompose. The second problem is statistical and deeply temporal: client data is non-IID and non-stationary. Each hospital's patients, each phone-owner's typing, each bank's customers follow a different and drifting distribution, exactly the non-stationarity that Chapter 20 studies, so the local optima pull in different directions and a plain average can converge slowly or to a poor compromise. This is why federated learning leans on personalization (a shared backbone with per-client heads) and on robust aggregation, and why the non-IID temporal setting remains an open research area rather than a solved one.

4. Synthetic Data and Membership-Inference Risk Intermediate

A third privacy strategy sidesteps both DP-SGD and federation: instead of protecting access to the real series, generate synthetic series that preserve the statistical structure useful for modeling while corresponding to no real person. The generative temporal models of Section 17.5, the GANs, diffusion models, and autoregressive samplers we built to create sequences, become privacy tools when their job is to produce a shareable surrogate dataset. A hospital can release synthetic vitals for a research challenge; a bank can publish synthetic transaction streams for a fraud-detection benchmark, and downstream users train on data that looks real in distribution but exposes no individual record. The appeal is that synthetic data composes freely with everything else: you can train any model on it, share it widely, and incur no per-query privacy cost.

The catch, and the reason synthetic data is a tool and not a guarantee, is that generators can memorize. A model that overfits its training sequences will regenerate them, near-verbatim, and a released synthetic dataset that contains a real patient's actual trajectory has leaked it completely, with a false sense of safety attached. The instrument that detects this is the membership-inference attack (MIA): given a candidate sequence and access to the model (or the synthetic output), the attacker decides whether that sequence was in the training set, typically by exploiting that models are more confident, or assign higher likelihood, to data they were trained on. A model is privacy-leaky to the exact degree that an MIA succeeds above chance. Membership inference is therefore the universal audit of this section: it tests DP-SGD models (a correct $\varepsilon$ provably caps MIA success), federated models (whose updates can be probed), and synthetic generators (whether the synthetic set betrays its training trajectories) on a common footing. The clean way to get a synthetic dataset with a real guarantee is to train the generator itself under DP-SGD, so that the formal $(\varepsilon, \delta)$ bound passes, by post-processing immunity, to every sample it ever produces.

5. Worked Example: DP-SGD and a Federated Round, From Scratch Then Library Advanced

We now make subsections two and three executable on a small temporal model and measure the accuracy cost of privacy. The plan mirrors this book's house pattern: build the mechanism by hand so its arithmetic is transparent, then reproduce it with the production library in a few lines, and read off the line-count reduction and the utility cost. We use a tiny GRU classifier over short synthetic sequences so the code runs in seconds; the mechanisms are identical at any scale. Code 33.4.1 is from-scratch DP-SGD: per-example gradient clipping, Gaussian noise, and a running budget.

import torch, torch.nn as nn, math

torch.manual_seed(0)
N, T, d_in, d_h, C = 256, 12, 3, 16, 1.0     # examples, len, in/hidden width, clip bound
X = torch.randn(N, T, d_in)
y = (X[:, :, 0].mean(1) > 0).long()           # label: is the first channel's mean positive

class GRUClf(nn.Module):
    def __init__(self):
        super().__init__()
        self.rnn = nn.GRU(d_in, d_h, batch_first=True)
        self.head = nn.Linear(d_h, 2)
    def forward(self, x):
        _, h = self.rnn(x)                    # h: (1, B, d_h) final hidden state
        return self.head(h.squeeze(0))

def per_example_grads(model, xb, yb):
    """Compute one clipped gradient PER example (the sensitivity unit)."""
    clipped = [torch.zeros_like(p) for p in model.parameters()]
    for i in range(len(xb)):                  # one sequence = one person here
        model.zero_grad()
        loss = nn.functional.cross_entropy(model(xb[i:i+1]), yb[i:i+1])
        loss.backward()
        g = [p.grad.detach().clone() for p in model.parameters()]
        norm = math.sqrt(sum((gi ** 2).sum().item() for gi in g))   # ||g_i||_2
        scale = min(1.0, C / (norm + 1e-12))  # clip to norm <= C: fixes sensitivity
        for acc, gi in zip(clipped, g):
            acc += gi * scale
    return clipped

def dp_sgd(sigma, epochs=30, lr=0.5, B=64):
    """Train with DP-SGD; return test accuracy. sigma is the noise multiplier."""
    model = GRUClf()
    for _ in range(epochs):
        idx = torch.randperm(N)[:B]           # Poisson-ish subsample (amplifies privacy)
        cg = per_example_grads(model, X[idx], y[idx])
        with torch.no_grad():
            for p, g in zip(model.parameters(), cg):
                noise = torch.randn_like(g) * sigma * C        # Gaussian, std = sigma*C
                p -= lr * (g + noise) / B                       # noised, averaged update
    with torch.no_grad():
        acc = (model(X).argmax(1) == y).float().mean().item()
    return acc

def epsilon_for(sigma, delta=1e-5):
    """Single-release Gaussian-mechanism epsilon for sensitivity C and this sigma."""
    return C * math.sqrt(2 * math.log(1.25 / delta)) / sigma    # invert sigma >= ... rule

for sigma in [0.0, 1.0, 4.84]:
    acc = dp_sgd(sigma)
    eps = float('inf') if sigma == 0 else epsilon_for(sigma)
    print(f"sigma={sigma:<4}  test_acc={acc:.3f}  per-step eps~={eps:.2f}")

sigma=0.0   test_acc=0.992  per-step eps~=inf
sigma=1.0   test_acc=0.945  per-step eps~=4.84
sigma=4.84  test_acc=0.797  per-step eps~=1.00

The same training would be reckless to hand-roll in production: the subtle parts are the privacy accountant and the efficient per-example gradients, both of which Opacus implements correctly. Code 33.4.2 reproduces DP-SGD with Opacus, which attaches to an ordinary optimizer and tracks a real Renyi-DP budget.

from opacus import PrivacyEngine               # pip install opacus
from torch.utils.data import TensorDataset, DataLoader

model = GRUClf(); opt = torch.optim.SGD(model.parameters(), lr=0.5)
loader = DataLoader(TensorDataset(X, y), batch_size=64, shuffle=True)

privacy_engine = PrivacyEngine()
model, opt, loader = privacy_engine.make_private(   # clipping + noise wired in here
    module=model, optimizer=opt, data_loader=loader,
    noise_multiplier=4.84, max_grad_norm=1.0)        # sigma and C, same as Code 33.4.1

for _ in range(30):
    for xb, yb in loader:
        opt.zero_grad()
        loss = nn.functional.cross_entropy(model(xb), yb)
        loss.backward(); opt.step()                  # per-example clip + noise, internally
eps = privacy_engine.get_epsilon(delta=1e-5)         # the composed, amplified budget
print(f"opacus trained, accountant eps={eps:.2f} at delta=1e-5")

Now the federated pair. Code 33.4.3 runs one FedAvg round from scratch: split the data across three non-IID clients, train each locally, and average the models weighted by client size, exactly the formula of subsection three.

import copy

def local_train(global_model, xb, yb, epochs=5, lr=0.5):
    """One client: copy the global model, train on LOCAL data only, return its weights."""
    m = copy.deepcopy(global_model)
    opt = torch.optim.SGD(m.parameters(), lr=lr)
    for _ in range(epochs):
        opt.zero_grad()
        nn.functional.cross_entropy(m(xb), yb).backward()
        opt.step()
    return m.state_dict(), len(xb)               # weights + example count n_k

# Non-IID split: client 0 mostly class 0, client 2 mostly class 1 (distribution shift)
order = y.argsort()                              # group by label to force non-IID shards
shards = [order[:90], order[90:170], order[170:]]
global_model = GRUClf()

updates = [local_train(global_model, X[s], y[s]) for s in shards]   # 3 clients train
total = sum(n for _, n in updates)
avg = copy.deepcopy(updates[0][0])
for key in avg:                                  # FedAvg: size-weighted parameter average
    avg[key] = sum(sd[key] * (n / total) for sd, n in updates)
global_model.load_state_dict(avg)
with torch.no_grad():
    acc = (global_model(X).argmax(1) == y).float().mean().item()
print(f"from-scratch FedAvg (1 round, 3 non-IID clients) acc={acc:.3f}")

from-scratch FedAvg (1 round, 3 non-IID clients) acc=0.781

Flower turns this hand-rolled round into a real client-server system (network transport, client sampling, secure aggregation, pluggable strategies) without changing the model. Code 33.4.4 is the Flower equivalent: the same local training and FedAvg, expressed against the framework's interfaces.

import flwr as fl                                # pip install flwr

class Client(fl.client.NumPyClient):            # wrap ONE client's local training
    def __init__(self, xb, yb):
        self.model, self.xb, self.yb = GRUClf(), xb, yb
    def get_parameters(self, config):
        return [p.detach().numpy() for p in self.model.parameters()]
    def set_parameters(self, params):
        for p, w in zip(self.model.parameters(), params):
            p.data = torch.tensor(w)
    def fit(self, params, config):              # server calls this each round
        self.set_parameters(params)
        sd, n = local_train(self.model, self.xb, self.yb)   # reuse Code 33.4.3's trainer
        self.model.load_state_dict(sd)
        return self.get_parameters(config), n, {}

# fl.simulation.start_simulation(client_fn=..., num_clients=3,
#     strategy=fl.server.strategy.FedAvg())     # FedAvg server-side; swap for DP/secure-agg

Read the four blocks together as the section's thesis in code. Code 33.4.1 built DP-SGD by hand and Output 33.4.1 priced privacy: the noisiest setting ($\sigma \approx 4.84$, a per-step $\varepsilon \approx 1$ whose composed model-level budget is larger) cost about twenty accuracy points on this toy task. Code 33.4.2 reproduced it with Opacus and a real accountant in a fraction of the lines. Code 33.4.3 ran FedAvg over non-IID temporal shards by hand, and Code 33.4.4 turned it into a deployable Flower system where DP and secure aggregation are strategy swaps. The pedagogical payoff is that neither mechanism is magic: DP-SGD is clip-then-noise with honest accounting, FedAvg is train-local-then-average, and the production libraries differ from the from-scratch versions mainly by correctness of the accountant and the realism of the transport, not by the core arithmetic you can now write yourself.

Exercises

1. (Conceptual.) Explain why providing record-level (per-window) differential privacy on a person's long sequence gives that person weaker protection than the reported $\varepsilon$ suggests. Describe how clipping per person-sequence instead of per window restores the intended user-level guarantee, and give one temporal scenario (for example a streaming model republished daily) where even user-level DP-SGD is insufficient and DP under continual observation is needed.
2. (Implementation.) Extend Code 33.4.3 to run twenty FedAvg rounds and plot global-model accuracy versus round for two data splits: an IID split (shuffle y before sharding) and the non-IID split shown. Confirm the non-IID curve converges more slowly and to a lower plateau, then add per-client DP noise (reuse the noising of Code 33.4.1 inside local_train) and report how much the privacy noise further slows convergence.
3. (Open-ended.) Train a small sequence generator (a GRU language model or the diffusion sampler of Section 17.5) on a temporal dataset twice, once normally and once under DP-SGD with Opacus, then implement a simple membership-inference attack (threshold on the model's per-sequence loss) against each. Report the attack's advantage over chance for both generators and discuss whether the DP $\varepsilon$ you trained at predicts the measured drop in attack success.