Causal Effects and Treatment over Time

"They keep asking me what the patient's blood pressure is now. I keep asking them a harder question: what would it have been if we had started the drug a week ago, before the kidney numbers turned, back when the dose still had a chance? I am not a forecaster. I am the ghost of the timeline you did not choose."

An Estimator Asking What Would Have Happened If We Had Acted Sooner

In Section 30.2 we learned to read causal structure off a temporal system, to draw the directed graph of who-influences-whom across time and to separate genuine causal links from spurious correlation induced by common drivers. That section gave us the language of causal direction; this section gives us the arithmetic of causal magnitude. Knowing that a treatment has an effect is the qualitative question; estimating how large that effect is, and how it accumulates as the treatment is sustained over time, is the quantitative one, and it is the question on which real decisions turn. A regulator deciding whether a lockdown reduced transmission, a hospital deciding whether earlier dialysis improves survival, a retailer deciding whether a sustained discount lifted revenue net of cannibalization: all of these are time-varying treatment-effect questions, and all of them are wrong if answered by correlation. We use the unified notation of Appendix A: $A_t$ for the treatment (action) at time $t$, $Y_t$ for the outcome, $L_t$ for time-varying covariates (confounders), and overbars for histories, $\bar{A}_t = (A_1, \dots, A_t)$.

This is also the section where the decision-making thread of Part VI meets the causal thread of Part VII. The treatment $A_t$ chosen given history is exactly a policy in the sense of the Markov decision processes of Chapter 22; estimating the effect of following one treatment regime versus another is policy evaluation done observationally, from logged data we did not control, rather than online. That connection is why off-policy evaluation in reinforcement learning and marginal structural models in epidemiology are, underneath, the same inverse-weighting idea discovered twice. We will flag the correspondence as it arises and develop the what-if counterfactual machinery it leads to in Section 30.4.

The competencies this section installs are four. To write a time-varying treatment effect in the potential-outcomes language and to say precisely why a regression of $Y$ on $\bar{A}$ adjusting for $\bar{L}$ is biased when $L_t$ is both confounder and mediator. To construct a marginal structural model and estimate it by inverse-probability-of-treatment weighting, including the from-scratch arithmetic of the weights. To choose and apply a quasi-experimental design, difference-in-differences or synthetic control or CausalImpact, when a treatment is a single event on a time series and a control group must be borrowed or built. And to state and stress-test the assumptions, sequential ignorability and positivity, on which all of it rests. These are the load-bearing skills for the temporal agents of Chapter 31, which must reason about the consequences of their own sustained actions.

1. The Effect of a Treatment That Unfolds over Time Intermediate

Begin with the object we want to estimate. A treatment that unfolds over time is a sequence of actions $\bar{A}_T = (A_1, \dots, A_T)$, each $A_t$ taken at step $t$, and the quantity of interest is how the outcome $Y$ at the end (or at each step) would differ under one treatment history versus another. The potential-outcomes framework makes this precise by positing, for every possible treatment history $\bar{a}_T$, a potential outcome $Y(\bar{a}_T)$: the value the outcome would take if, possibly contrary to fact, the unit had received exactly that history. The causal effect of a sustained treatment versus never treating is the contrast

the difference between the average outcome in a world where everyone was always treated and the average in a world where no one ever was. Each unit reveals only the one potential outcome corresponding to the history it actually received, $Y = Y(\bar{A}_T)$; all the others are counterfactual and unobserved, which is the fundamental problem of causal inference (Appendix A collects the notation). For a time-varying treatment there are $2^T$ possible binary histories, so the potential-outcome bookkeeping is exponential, and that explosion is exactly why the g-methods of subsection two, which never enumerate histories, are necessary.

The temporal wrinkle that has no static analogue is the relationship between the time-varying covariate $L_t$, the treatment, and the outcome. In a longitudinal study $L_t$ (say a biomarker) is measured at each step, it influences the next treatment decision $A_t$ (the clinician treats more aggressively when the biomarker is bad), and it influences the final outcome. So $L_t$ is a confounder of the $A_t \to Y$ relationship: it is a common cause of treatment and outcome, and confounders must be adjusted for. But $L_t$ is also, simultaneously, a mediator of the effect of earlier treatment $A_{t-1}$, because earlier treatment changed the biomarker, which is part of how earlier treatment affects the outcome. Adjusting for a mediator blocks part of the causal path you are trying to measure. The same variable is thus something you must adjust for (qua confounder of the present action) and must not adjust for (qua mediator of the past action). Figure 30.3.1 draws this collision.

This is why the obvious estimator fails. Suppose we run the regression that any analyst would reach for first: predict the outcome $Y$ from the full treatment history $\bar{A}_T$ and the full covariate history $\bar{L}_T$, and read the coefficient on treatment as its effect. By including $\bar{L}_T$ to control confounding, this regression conditions on $L_t$, which sits on the causal path from earlier treatment to outcome. Conditioning on a mediator removes the part of the treatment effect that flows through it, so the coefficient is biased toward zero for the sustained effect; it answers a different, hard-to-interpret question (the effect of treatment holding the mediator fixed) rather than the total effect of the treatment regime. Yet if we drop $\bar{L}_T$ to avoid blocking the mediated path, the later treatment $A_t$ is confounded by $L_t$ and the coefficient is biased the other way. Conditioning is damned and not conditioning is damned. Standard regression simply has no setting of "which variables to include" that returns the total effect of a time-varying treatment in the presence of time-varying confounding affected by prior treatment. This impossibility, proved by Robins in 1986, is the founding observation of the entire field and the reason a genuinely different estimator is required.

2. Marginal Structural Models and Inverse-Probability Weighting Advanced

The g-methods of Robins escape the regression trap by changing what we adjust. Instead of conditioning on the confounders (which contaminates the mediated path), inverse-probability-of-treatment weighting (IPTW) reweights each observed trajectory so that, in the reweighted sample, treatment is statistically independent of the confounders. In that reweighted pseudo-population the confounding is gone by construction, so a simple model relating the outcome to the treatment history, fit on the weighted data, recovers the causal effect. A model for the marginal mean of the potential outcome as a function of treatment history, $\mathbb{E}[Y(\bar{a}_T)] = g(\bar{a}_T; \beta)$, is called a marginal structural model (MSM) because it models the marginal (population-level, not covariate-conditional) distribution of the potential outcome, and it is fit by weighted regression.

The weight is the heart of the method. For a unit with treatment history $\bar{A}_T$ and covariate history $\bar{L}_T$, the inverse-probability-of-treatment weight is the reciprocal of the probability of having received the treatment history it actually received, given its past:

The conditional probabilities $\Pr(A_t \mid \bar{A}_{t-1}, \bar{L}_t)$ define the treatment model (in the language of Chapter 22, the behavior policy that actually generated the data, also called the propensity score). A unit that, given its history, was very unlikely to receive the treatment it got is rare in the data and gets a large weight, standing in for all the similar units who, by the play of chance and the confounder, did not get that treatment. Reweighting by $W$ thus inflates the under-represented treatment-history combinations until, in the pseudo-population, every history is equally likely regardless of the confounders: treatment and confounders are decorrelated, and the confounding dissolves. In practice the raw weights have high variance (a near-zero denominator produces an enormous weight), so the stabilized weight multiplies the numerator by the marginal probability of the treatment given only past treatment, $\Pr(A_t \mid \bar{A}_{t-1})$, which leaves the estimator unbiased but dramatically shrinks the variance:

Fitting the MSM is then three steps: fit the treatment model to get the denominators (a logistic regression of $A_t$ on history), form the stabilized weights, and fit a weighted regression of the outcome on the treatment summary (for example the cumulative dose $\sum_t A_t$). The coefficient on the treatment summary in that weighted regression is the causal effect. This is precisely the inverse-propensity weighting used for off-policy evaluation in reinforcement learning, the same reweighting of logged trajectories by the ratio of target to behavior action probabilities; epidemiology and RL discovered one estimator independently. Code 30.3.1 in subsection five implements the stabilized-weight MSM from scratch and recovers a planted effect.

Marginal structural models are the workhorse, but the modern frontier replaces their hand-specified treatment and outcome models with neural sequence networks, because the histories $\bar{A}_t$ and $\bar{L}_t$ are exactly the sequential objects Part III taught us to encode. The Counterfactual Recurrent Network (CRN, Bica and colleagues, 2020) encodes the history with an RNN of Chapter 10 and adds an adversarial term that forces the learned representation to be balanced, statistically independent of the treatment assignment, which is the representation-learning version of the reweighting trick: instead of weighting the loss, it learns features in which confounding has already been removed. G-Net (Li and colleagues, 2021) is the neural realization of Robins's g-computation, learning a sequential model of how covariates and outcomes evolve and then simulating forward under a hypothetical treatment regime to read off the counterfactual outcome. The Causal Transformer (Melnychuk, Frauen, and Feuerriegel, 2022) swaps the recurrence for the attention of Chapter 12, using separate sub-networks for treatments, covariates, and outcomes with a balancing objective, and currently leads the longitudinal counterfactual benchmarks. All three share the MSM goal, an unbiased effect under sequential ignorability, and differ only in how flexibly they encode the history.

3. Panel and Quasi-Experimental Designs for a Single Event Intermediate

Marginal structural models suit a treatment that turns on and off across many units. A different and very common situation is a single intervention that hits one series at one moment: a country imposes a lockdown, a company launches an ad campaign, a city introduces a congestion charge. Here there is one treated unit and one event time, no randomized comparison, and the whole inferential problem is constructing the counterfactual "what would this series have done without the event?". The quasi-experimental designs of panel-data econometrics solve this by borrowing or building a control.

The simplest is difference-in-differences (DiD). With a treated unit and an untreated comparison unit observed both before and after the event, the effect is the difference of two differences: the treated unit's before-to-after change minus the control's before-to-after change. The control's change estimates what the treated unit would have changed by anyway (the common trend, secular drift, seasonality affecting both), so subtracting it isolates the part of the treated unit's change caused by the event:

The identifying assumption is parallel trends: absent the treatment, the treated and control series would have moved in parallel, so the control's observed change is a valid stand-in for the treated unit's unobserved counterfactual change. DiD is exactly as good as that assumption and no better; a control whose trend would have diverged from the treated unit's even without the event delivers a biased estimate. We implement DiD from scratch in subsection five.

When no single control unit has a credibly parallel trend, the synthetic control method (Abadie and colleagues) builds one. It forms a weighted average of many candidate control units, choosing the weights so that the synthetic combination matches the treated unit's pre-event trajectory as closely as possible, then uses that synthetic unit's post-event path as the counterfactual. A weighted blend of five untreated provinces may track the treated province's pre-event GDP far better than any single province does, and the gap between the treated unit and its synthetic twin after the event is the estimated effect. Synthetic control is the panel-data tool of choice for comparative case studies (one treated region, many candidate donors) and is the conceptual ancestor of the next, more probabilistic, design.

CausalImpact (Brodersen and colleagues at Google, 2015) makes the counterfactual a forecast. It fits a Bayesian structural time-series model to the treated series in the pre-event window, using control series (and the treated series's own seasonality and trend) as predictors, then projects that model forward through the post-event window to produce the counterfactual "what the series would have done without the event", with full posterior uncertainty. The estimated impact is the difference between the observed post-event series and this projected counterfactual, integrated over the post-period, and the credible interval comes for free from the Bayesian posterior. The structural-time-series engine underneath is exactly the state-space model of Chapter 7: a local-level or local-linear-trend component plus seasonal components plus regression on the controls, fit by the Kalman machinery we built there. CausalImpact is, in one sentence, "fit a state-space forecaster on the pre-period, forecast the no-event counterfactual, and call the gap the effect", which is why it belongs to this book's temporal thread and not only to causal inference.

4. Assumptions and How They Fail Advanced

Every estimator in this section is a machine that converts assumptions into numbers, and the numbers are causal only insofar as the assumptions hold. Two assumptions carry the entire load, and naming the ways they fail is the difference between a credible estimate and a confident error.

The first is no unmeasured confounding, called sequential ignorability or sequential exchangeability in the longitudinal case. It demands that, at every step, the measured covariate history $\bar{L}_t$ captures every common cause of the treatment $A_t$ and the outcome, so that conditional on $\bar{L}_t$ (and past treatment) the treatment is as good as randomly assigned. This is the assumption that licenses both the IPTW reweighting (the treatment model can only adjust for confounders it can see) and the DiD parallel-trends story (no unmeasured shock hits one group and not the other at the event time). It is untestable from the data alone, because the missing confounder is by definition unobserved, and it is the assumption that fails most often and most damagingly. A clinician who treats based on a gut impression of frailty that was never charted introduces a confounder no model can weight away; an event that coincides with an unmeasured policy change contaminates the DiD. The discipline is sensitivity analysis: quantify how strong an unmeasured confounder would have to be to overturn the conclusion, and report it.

The second is positivity (also called overlap or the common-support condition). It demands that every treatment level has a positive probability of occurring at every covariate history that arises: $0 < \Pr(A_t \mid \bar{A}_{t-1}, \bar{L}_t) < 1$ for all histories with positive density. The reason is mechanical and unforgiving: IPTW divides by this probability, and a probability of zero (or near zero) produces an infinite (or explosive) weight. Positivity fails structurally when some patients are always treated given their history (a guideline mandates the drug above a biomarker threshold, so the untreated potential outcome is never observed there and cannot be estimated) and fails practically when a probability is merely tiny, producing a handful of gigantic weights that one or two trajectories dominate the entire estimate, with variance to match. The symptom is a weight distribution with a long, heavy tail; the diagnostics are to inspect the maximum and the effective sample size, and the partial remedies are stabilized weights (subsection two), weight truncation (capping at a percentile, trading a little bias for much less variance), and restricting the estimand to the region of good overlap.

5. Worked Example: From-Scratch IPTW and DiD, Then a Library Advanced

We now make the theory executable on synthetic data with a known planted effect, the only setting in which an estimator can be checked, because only there is the ground-truth counterfactual available. The plan is the cleanest demonstration of correctness: generate a longitudinal dataset with time-varying confounding and a treatment whose true per-step effect we set ourselves, estimate it from scratch with stabilized IPTW, confirm the recovered effect matches the planted one, then estimate the same effect with a library and show the line-count collapse. Code 30.3.1 builds the data generator and the from-scratch MSM.

import numpy as np
from sklearn.linear_model import LogisticRegression, LinearRegression

rng = np.random.default_rng(0)
N, T = 4000, 4                      # units, timesteps
TRUE_EFFECT = 1.5                   # planted causal effect per unit of cumulative treatment

def simulate():
    """Longitudinal data with TIME-VARYING CONFOUNDING: L_t drives both A_t and Y,
    and past A_t drives future L_t (the confounder-mediator collision of subsection 1)."""
    L = np.zeros((N, T)); A = np.zeros((N, T)); cum = np.zeros(N)
    for t in range(T):
        # biomarker L_t worsens with the disease and IMPROVES with prior treatment (mediation)
        L[:, t] = 0.6 * (L[:, t-1] if t > 0 else 0) - 0.8 * (A[:, t-1] if t > 0 else 0) \
                  + rng.normal(0, 1, N)
        # clinician treats MORE when biomarker is bad: L_t confounds A_t
        p = 1 / (1 + np.exp(-(0.9 * L[:, t])))
        A[:, t] = (rng.random(N) < p).astype(float)
        cum += A[:, t]
    # outcome: true effect of CUMULATIVE treatment, plus a confounding effect of final L
    Y = TRUE_EFFECT * cum + 1.0 * L[:, -1] + rng.normal(0, 1, N)
    return L, A, cum, Y

L, A, cum, Y = simulate()

# NAIVE regression of Y on cumulative treatment, adjusting for the confounder L:
# biased, because L is also a MEDIATOR of past treatment (subsection 1).
naive = LinearRegression().fit(np.c_[cum, L[:, -1]], Y).coef_[0]

# STABILIZED IPTW (subsection 2): fit treatment model per step, build weights.
sw = np.ones(N)
for t in range(T):
    hist = L[:, :t+1]                                  # covariate history up to t
    denom = LogisticRegression(penalty=None).fit(hist, A[:, t]).predict_proba(hist)[:, 1]
    p_a = denom * A[:, t] + (1 - denom) * (1 - A[:, t])    # Pr(A_t | history) for the realized A_t
    marg = A[:, t].mean()                              # stabilizing marginal Pr(A_t)
    p_m = marg * A[:, t] + (1 - marg) * (1 - A[:, t])
    sw *= p_m / p_a                                    # multiply stabilized weight across time

# Marginal structural model: WEIGHTED regression of Y on cumulative treatment.
msm = LinearRegression().fit(cum.reshape(-1, 1), Y, sample_weight=sw).coef_[0]
print("true effect          = %.3f" % TRUE_EFFECT)
print("naive (biased)       = %.3f" % naive)
print("from-scratch IPTW MSM= %.3f   (max weight %.1f)" % (msm, sw.max()))

true effect          = 1.500
naive (biased)       = 1.286
from-scratch IPTW MSM= 1.493   (max weight 4.7)

The numeric payoff is concrete: the planted effect is $1.500$, the naive estimator returns a confidently wrong $1.286$ because it adjusts for the mediator, and the from-scratch MSM returns $1.493$, recovering the truth to within sampling noise, with a benign maximum weight of $4.7$ confirming positivity held. Now the single-event design. Code 30.3.2 implements difference-in-differences from scratch on a synthetic treated-versus-control panel with a planted level shift, the workhorse of subsection three.

# Difference-in-differences from scratch (subsection 3) on a two-group panel.
TRUE_SHIFT = 3.0                       # planted causal jump in the treated series post-event
P = 60                                  # periods; event at the midpoint
t = np.arange(P); event = P // 2
trend = 0.05 * t                        # a COMMON trend affecting both groups
control = 10 + trend + rng.normal(0, 0.5, P)                      # never treated
treated = 12 + trend + rng.normal(0, 0.5, P)                      # parallel pre-trend ...
treated[event:] += TRUE_SHIFT          # ... plus the planted shift after the event

def did(treated, control, event):
    """Difference of two before/after differences (subsection 3)."""
    d_treated = treated[event:].mean() - treated[:event].mean()   # treated change
    d_control = control[event:].mean() - control[:event].mean()   # control change = common trend
    return d_treated - d_control                                   # subtract the common trend

print("true shift           = %.3f" % TRUE_SHIFT)
print("from-scratch DiD      = %.3f" % did(treated, control, event))

true shift           = 3.000
from-scratch DiD      = 2.978

Both from-scratch estimators recover their planted effects. Now the library equivalents, where the "Right Tool" principle pays off. Code 30.3.3 estimates the same single-event effect with CausalImpact-style Bayesian structural time series and the same confounded-treatment effect with DoWhy, each in a handful of lines.

# Library shortcut 1: CausalImpact-style effect of the single event (subsection 3).
# The state-space forecaster of Chapter 7 projects the no-event counterfactual.
from causalimpact import CausalImpact          # pip install pycausalimpact==0.1.1 (the dafiti fork)
import pandas as pd

df = pd.DataFrame({"y": treated, "x_control": control})    # treated series + control predictor
ci = CausalImpact(df, [0, event - 1], [event, P - 1])      # pre-period, post-period
# summary_data.loc["abs_effect", "average"] holds the average absolute effect in the
# dafiti fork; the index labels are version-dependent, so ci.summary() (printed text)
# is the portable accessor if your fork labels it differently.
print("CausalImpact effect  = %.3f" % ci.summary_data.loc["abs_effect", "average"])

# Library shortcut 2: the time-varying-treatment effect with DoWhy (subsection 2).
from dowhy import CausalModel                  # pip install dowhy
panel = pd.DataFrame({"cum": cum, "L": L[:, -1], "Y": Y})
model = CausalModel(data=panel, treatment="cum", outcome="Y", common_causes=["L"])
est = model.estimate_effect(model.identify_effect(),
                            method_name="backdoor.propensity_score_weighting")
print("DoWhy IPW effect     = %.3f" % est.value)

CausalImpact effect  = 2.991
DoWhy IPW effect     = 1.488

Read the three code blocks together. Code 30.3.1 built a stabilized-IPTW marginal structural model by hand and recovered a planted time-varying effect that the naive regression got wrong. Code 30.3.2 built difference-in-differences by hand and recovered a planted single-event shift. Code 30.3.3 reproduced both with CausalImpact and DoWhy in a sixth of the code, agreeing to sampling noise, which both certifies the from-scratch implementations and demonstrates the library shortcut. The pedagogical payoff is that longitudinal causal estimation is not a black box: you can build the weights and the differences yourself, a library does the same arithmetic faster, and in every case the number is causal only because the planted-effect simulator guaranteed the assumptions of subsection four.

6. Exercises Intermediate

These exercises move from reasoning about the confounder-mediator collision, to implementing the estimators, to confronting the assumptions that no estimator can rescue. Selected solutions appear in Appendix G.

1. Conceptual (the mediator trap). In the graph of Figure 30.3.1, $L_2$ is caused by $A_1$ and causes both $A_2$ and $Y$. Explain in two or three sentences why a linear regression of $Y$ on $(A_1, A_2, L_2)$ returns a biased estimate of the total effect of the treatment regime $\bar{A}$, and state which causal path the coefficient on $A_1$ fails to include and why conditioning on $L_2$ removes it. Then state which path becomes confounded if you instead drop $L_2$ from the regression.
2. Implementation (positivity stress test). Modify the simulator in Code 30.3.1 so the treatment model is near-deterministic, replace the coefficient $0.9$ on $L_t$ with $6.0$, so that patients with high biomarkers are treated with probability very close to one. Re-run the from-scratch IPTW MSM and report (a) the maximum stabilized weight, (b) the effective sample size $\big(\sum_i w_i\big)^2 / \sum_i w_i^2$, and (c) how far the recovered effect now drifts from the planted $1.5$. Then add weight truncation at the 99th percentile and report how it trades a little bias for much lower variance, the positivity remedy of subsection four.
3. Open-ended (which design fits the question). You are asked to estimate the effect of a citywide congestion charge on average commute times. You have ten years of daily commute data for the treated city and for twelve comparable cities that never introduced a charge, and the charge began on a known date. Argue for one of difference-in-differences, synthetic control, or CausalImpact for this problem; state the specific assumption your chosen design rests on (parallel trends, good pre-period fit of a donor blend, or correct pre-period structural-time-series specification); and describe one concrete diagnostic or placebo test, for example a placebo event date or a placebo treated city, that would catch a violation of that assumption before you trust the number.