Capstone
Capstone Project

Capstone Project: An End-to-End Temporal AI System

Design, build, evaluate, and operate a complete temporal system that spans the whole book.

"Thirty-six chapters of theory, and now they want me to predict next week, decide what to do about it, and stay calibrated when the world drifts. I was afraid they would ask. I was more afraid they would not."

A Capstone Project That Finally Uses Every Chapter
The Big Picture

Every chapter of this book taught one capability in isolation: a way to clean a stream, a model to fit, an interval to calibrate, a policy to learn, a drift detector to watch. The capstone removes the isolation. You will take a single dataset from raw observations all the way to a system that makes a decision, knows how confident it is, and keeps working after the world changes underneath it. The goal is not a notebook with one good number; it is a small but honest system that a colleague could rerun, inspect, and trust. By the time you are done, the temporal thread that ran through the book, the Kalman filter becoming the recurrent network, the forecast becoming a decision, the offline policy becoming a deployed agent, will run through a project that is entirely your own.

1. The Brief

Build an end-to-end temporal AI system on a dataset of your choice. You may use one of the book's three running domains (a finance returns-and-volatility series, an irregularly sampled multivariate clinical series, or an industrial sensor-and-telemetry stream), or you may bring your own data, provided it is genuinely temporal in the sense of Chapter 1: the order carries the signal, the past constrains the future, and shuffling the rows would destroy what you are trying to learn. The system must travel the full distance from raw, messy, unaligned data to a deployed artifact that drives a decision, reports calibrated uncertainty, and is monitored for the day its assumptions stop holding.

The deliverable is a system, not a result. A model that scores well on a single split but cannot be rerun, cannot say how sure it is, and turns its forecast into nothing actionable has missed the point of the entire book. The capstone is deliberately weighted toward the parts that academic exercises usually skip: the leakage discipline that separates a real number from an inflated one, the decision layer that gives the forecast a reason to exist, and the operational plan that keeps the system honest after it ships. A modest model wired correctly into a complete, reproducible, decision-driving pipeline outscores a state-of-the-art model that lives alone in a notebook.

Key Insight

The capstone is graded on the whole loop, not the peak metric. The chain is only as strong as its weakest link, and in temporal systems the weakest link is almost never the architecture. It is the chronological split that quietly leaked, the interval that was never checked for coverage, the forecast that nobody turned into an action, or the model that silently rotted three weeks after deployment. Spend your effort where the book spent its warnings.

2. The Required Stages

Every capstone, whichever track you choose, must pass through the seven stages below. Each stage names the chapters that equip it, so the capstone doubles as a guided tour back through the book. Treat the stages as a pipeline: the output of one is the audited input of the next, and a failure upstream (a leak in stage one) invalidates everything downstream no matter how strong it looks.

2.1 Data Engineering and a Leakage-Free Pipeline (Part I)

Begin where Chapter 2 begins. Resample and align your streams without inventing the future, build features that respect the arrow of time, handle missing values and irregular sampling honestly, and construct windows and a strictly chronological train, validation, and test split. The single non-negotiable here is leakage discipline: no feature may use information unavailable at prediction time, no normalization statistic may be fit on the test segment, and no split may place a test point before a training point in wall-clock time. Document the pipeline as code that runs start to finish from the raw files, because everything downstream inherits its correctness.

2.2 A Strong Classical Baseline (Part II)

Before any deep model, fit a real classical baseline from Part II: a seasonal-naive or exponential-smoothing reference at minimum, and ideally a properly fitted ARIMA, GARCH, or state-space model (Chapter 5, Chapter 6, Chapter 7) chosen to fit your data type. The baseline is not a formality; it is the bar. A strong classical model is often startlingly hard to beat, and the entire value of the next stage is measured against this number. A weak baseline is a way of lying to yourself about how good your deep model is.

2.3 A Deep or Foundation Model That Beats the Baseline (Part III)

Now bring a model from Part III: a recurrent network, a temporal convolutional network, a Transformer, a structured state-space model, or a temporal foundation model used zero-shot or fine-tuned (Chapter 10 through Chapter 15). The requirement is sharp: this model must beat the classical baseline of stage 2.2 on the held-out test segment, on a metric you fixed in advance, by a margin you can defend as more than noise. If it does not, that is a finding, and the disciplined response is to report the baseline as your production model and explain why the deep model did not earn its complexity. A foundation model that wins zero-shot is a perfectly good answer; so is a small TCN that wins by being well-regularized.

2.4 Calibrated Uncertainty and Conformal Intervals (Chapter 19)

A point forecast is not a deliverable. Wrap your chosen model in calibrated uncertainty using the methods of Chapter 19: produce prediction intervals and verify their empirical coverage on the test segment, then tighten them with conformal prediction so that a nominal ninety percent interval actually contains the truth about ninety percent of the time. Report coverage and average interval width together, because an interval that is always right by being always wide is useless. Calibration is what lets the decision layer downstream trust the numbers it is handed.

2.5 From Forecast to Decision, and the Value of the Decision (Chapter 19.6 / Part VI)

This is the stage that gives the forecast a reason to exist. Turn the calibrated prediction into an actual decision (Section 19.6 and the sequential-decision machinery of Part VI): a position size, a treatment trigger, a maintenance order, an inventory quantity, a route. Then evaluate the decision, not just the forecast. Define a decision value (profit, cost avoided, regret, expected utility, lives or downtime saved) and measure it against the decision your baseline would have made. A forecast that improves mean absolute error but degrades the downstream decision is a failure that only the decision layer can reveal, which is exactly why the book insists on it.

2.6 Monitoring, Drift Detection, and a Retraining and Serving Plan (Chapters 20, 21, 34)

Plan for the system's life after deployment. Using Chapter 20, Chapter 21, and Chapter 34, specify how you will detect distribution shift and concept drift, what triggers a retrain, how the model is served, and how you roll a new version forward without a gap. Demonstrate the monitoring on a held-out tail of the data engineered to drift (or a naturally non-stationary stretch), and show your detector firing and your retraining policy responding. A system that cannot notice it has gone stale is a liability, not an asset.

2.7 A Trustworthiness Review (Chapter 33)

Finally, audit the system for trustworthiness with Chapter 33. Explain what the model attends to and why it makes the decisions it makes; check robustness to plausible perturbations and outliers; and where your domain demands it, assess fairness across groups and any privacy exposure in the data or model. A clinical or financial capstone that cannot say why it recommended an intervention, or that quietly disadvantages a subpopulation, fails this stage regardless of its accuracy. Trustworthiness is a stage, not a sentence in the conclusion.

The Temporal Thread, End to End

Notice that the seven stages retrace the whole arc of the book in miniature. The classical baseline of stage 2.2 is the same idea that returns, in learned form, as the deep model of stage 2.3, exactly the Kalman-filter-becomes-RNN thread from Chapter 1. The forecast of stage 2.4 becomes the decision of stage 2.5, the prediction-to-decision thread that organizes Parts V and VI. Building the capstone is the fastest way to feel that classical analysis and modern temporal deep learning are one subject at two magnifications.

3. Three Suggested Project Tracks

You may design your own project, but most readers do best starting from one of the three tracks below. Each fixes a domain, a decision, and a value metric, so you can begin engineering immediately rather than spending a week choosing. All three pass through the same seven stages of Section 2; they differ in what the decision is and how its value is scored.

3.1 Forecasting and Decision Track

Forecast a finance returns-and-volatility series or an energy-demand series, calibrate the forecast, and turn it into a position or a procurement quantity. The deep model (a Transformer or a foundation model such as Chronos or TimesFM, used zero-shot then fine-tuned) must beat a GARCH or seasonal-ARIMA baseline. The decision value is risk-adjusted profit or procurement cost, evaluated with realistic transaction or holding costs, against the baseline's decisions. The lesson of this track is that lower forecast error and higher decision value are not the same thing, and the gap between them is where the money is.

3.2 Anomaly and Predictive-Maintenance Track

Monitor an industrial sensor-and-telemetry stream for anomalies and impending failures (Chapter 8, Chapter 32). The deep detector must beat a classical change-point or residual-based baseline on detection latency at a fixed false-alarm rate. The decision is when to schedule maintenance; the value is downtime and cost avoided minus the cost of false alarms. Drift monitoring is the natural centerpiece here, since real telemetry drifts as equipment ages, so stage 2.6 is where this track earns most of its grade.

3.3 Sequential-Decision and Reinforcement-Learning Track

Treat the problem as control rather than forecasting (Chapter 22 through Chapter 29): a clinical treatment policy, an inventory or bidding agent, or a robotic controller learned from offline logs with offline RL (d3rlpy) or as a Decision Transformer (Chapter 28). The learned policy must beat a sensible scripted or behavior-cloning baseline on cumulative reward in a held-out evaluation. The classical baseline here is the behavior policy in the logs; the uncertainty stage becomes off-policy value estimation with confidence bounds, and the trustworthiness review is especially load-bearing when the decisions affect people.

4. Deliverables

The capstone produces exactly three artifacts. Keep them lean; an examiner should be able to verify the whole project in an afternoon.

5. Assessment Rubric

The capstone is assessed along seven dimensions. The table below states what excellent work looks like on each and the common pitfall that sinks each. Read it before you start, not after; almost every row describes a mistake that is cheap to avoid early and expensive to fix late. The leakage and decision-awareness rows carry the most weight, because they are where temporal systems most often fail silently.

Dimension What Excellent Looks Like Common Pitfall
Correctness Metrics computed once, on one fixed test segment, with a pre-registered loss; every number traceable to a line of code. Cherry-picked best run across many seeds; metrics from mismatched configs compared as if commensurable.
Leakage discipline Strictly chronological splits; all statistics fit on training only; features provably available at prediction time; a documented leakage audit. Random shuffle split; normalization fit on the full series; a target-derived feature; future information in a lag computed across the split boundary.
Baseline-beating A strong classical baseline is properly tuned, and the deep model beats it by a defensible margin on the held-out segment. A deliberately weak baseline (or none) so the deep model looks good; a margin within the noise reported as a win.
Uncertainty Conformal intervals with verified empirical coverage near nominal, reported with average width; calibration shown, not asserted. Point forecasts only; intervals whose coverage is never checked; intervals widened until trivially correct.
Decision-awareness The forecast drives a concrete decision; decision value is defined and beats the baseline's decisions under realistic costs. The forecast is the end product; lower error reported while the downstream decision is never simulated or scored.
Reproducibility One command regenerates every figure from raw data on a clean machine; seeds, versions, and config all pinned and logged. Manual undocumented steps; unpinned dependencies; hard-coded paths; numbers in the report that no longer reproduce.
Trustworthiness The model is explained, stress-tested for robustness, and audited for fairness and privacy where the domain demands it. A black box shipped without interpretation; no robustness check; group disparities or privacy leakage left unexamined.

The orienting skeleton below shows how the seven stages compose into a single runnable pipeline. It is intentionally schematic; your real code will replace each call with the methods from the cited chapters, but the shape (audit, baseline, beat, calibrate, decide, monitor, audit again) should survive. 

def run_capstone(raw):
    data = build_leakage_free_pipeline(raw)        # Part I: align, window, chrono-split
    base = fit_classical_baseline(data.train)      # Part II: ARIMA / GARCH / state-space
    model = fit_deep_model(data.train, data.val)   # Part III: RNN / TCN / Transformer / FM
    assert beats(model, base, data.test)           # stage 2.3: must beat the baseline

    intervals = conformalize(model, data.cal)      # Ch 19: calibrated coverage
    assert empirical_coverage(intervals, data.test) >= 0.89

    decisions = forecast_to_decision(model, intervals)        # Sec 19.6 / Part VI
    value = decision_value(decisions, data.test)              # score the decision, not the point
    assert value > decision_value(baseline_decisions(base), data.test)

    monitor = DriftMonitor(retrain=retrain_policy)            # Ch 20, 21, 34
    audit = trustworthiness_review(model, decisions, data)    # Ch 33
    return model, intervals, decisions, monitor, audit
A schematic capstone pipeline: each call stands in for the methods of its cited chapters, and the assertions encode the rubric's non-negotiables (beat the baseline, verify coverage, beat the baseline's decisions).

6. Milestone Timeline and Tips

Spread the work across five milestones rather than one final sprint, because the upstream stages gate the downstream ones and a leak found late forces a full rerun. The suggested rhythm below assumes a project carried over several weeks of part-time effort; compress or stretch it to your schedule, but keep the order.

Three Tips That Save Whole Weekends

First, fix your test segment and your metric on day one and quarantine them; every time you peek, you spend a little of your honesty. Second, build the decision layer earlier than feels necessary, because it routinely overturns the model you thought you were shipping and you want that surprise in week three, not week eight. Third, write the reproducibility script as you go, not at the end; a pipeline that has always rerun from raw data is trivial to keep that way, and a pipeline that never has is agony to fix.

Research Frontier: Where an Ambitious Capstone Can Push

The strongest capstones touch the live edge of the field. Zero-shot foundation forecasters (Chronos, Moirai, TimesFM, Lag-Llama) now sometimes beat tuned task-specific models with no training at all, so a foundation-model-as-baseline comparison is genuinely open research. Conformal prediction under distribution shift (conformal PID, adaptive conformal inference) is an active area where your drift-and-recalibrate loop could contribute a real result. And the Decision-Transformer line, recasting control as conditional sequence modeling, makes the forecasting and reinforcement-learning tracks two views of one model. A capstone that reports a careful, leakage-clean comparison on any of these is publishable-shaped work, not a class exercise.

A Note on Confidence

The model you finish with will be less impressive than the one you imagined and more trustworthy than the one you feared. That trade is the whole job. A forecaster confident about next Tuesday is charming until Tuesday; a system that says how unsure it is, decides accordingly, and notices when it has gone stale is the one you would actually put in front of a decision that matters.