"For seventeen chapters they fed me values on a tidy grid, one number per tick, the ticks evenly spaced, and asked me only what the next number would be. Then someone showed me a hospital ward, and the grid dissolved. A patient's vitals arrive when a nurse happens to look, an alarm fires at 3:14 and again at 3:16 and then not for an hour, a transfer order lands out of nowhere. Nothing was evenly spaced anymore, and worse, the spacing itself was the message: two alarms in two minutes mean something a single alarm an hour apart does not. I had to stop asking what value comes next and start asking when the next thing happens at all, and what kind of thing it will be, and whether it ever happens before the record simply ends. The clock was no longer the axis I plotted against. The clock had become the data."
A Point Process That Finally Noticed the Gaps Between the Events
Chapter Overview
Every model so far in this book has assumed a quiet luxury: that your data arrives as values on an even grid, one observation per regular tick, so that the only question worth asking is what value comes at the next tick. Vast amounts of the most consequential temporal data do not look like that at all. A patient's clinical record is a stream of events that occur when something happens, a clinic visit, a lab result, a medication order, a transfer, each stamped with the moment it occurred and a label saying what kind of event it was. A trade tape is a torrent of buy and sell events at irregular microsecond-spaced times. A user's interaction log, a server's error stream, a network of accounts transacting, all are sequences of marked events in continuous time, and for them the timing itself carries the information. The gap between two events, the burst, the long silence, these are not nuisances to be resampled away onto a grid; they are the signal. This chapter is about modeling that signal directly, treating time as a continuous variable that events are scattered across rather than an axis you read values off of.
The chapter takes the territory in four movements. It opens with the data type itself, the marked event stream, and the vocabulary for describing it: events, marks, inter-event times, and the conditional intensity function that says how likely an event is to fire right now given everything that came before. From there it builds the central model family, the temporal point process, starting with the classical self-exciting Hawkes process, in which each event raises the chance of more events soon after, the way a tweet provokes retweets or a trade provokes trades, and then moving to its neural descendants, the recurrent and attention-based neural temporal point processes that learn the intensity from data rather than assuming a fixed parametric form. The third movement lifts events onto a graph, where the entities themselves form a network that changes over time, and temporal and dynamic graph models like TGAT and TGN learn representations of nodes whose neighborhoods evolve event by event. The fourth movement turns to the questions practitioners most often actually ask of event data: when will the next event happen, and will a particular event of interest, a failure, a churn, a death, happen at all, which is the domain of event prediction and time-to-event survival analysis with its careful handling of censored records. A closing movement steps up to whole processes, mining the structure of a business or clinical workflow from its event log, the discipline of process mining.
The running datasets make the shift concrete. The healthcare thread of the book, the irregularly sampled clinical record introduced in Chapter 2 and filtered in Chapter 7, returns here in its native form as a marked event stream, and time-to-event modeling of patient outcomes is exactly the survival analysis this chapter teaches. The finance thread reappears as an order or transaction stream where a Hawkes process captures the self-exciting clustering of activity. Across both, the lesson is the same: when you stop forcing events onto a grid and model their timing directly, you recover information that grid-based forecasting throws away.
This chapter completes Part IV, the representation-learning arc of the book. Chapter 16 learned representations of regular series without forecasting labels, through self-supervised and contrastive objectives; Chapter 17 learned to generate temporal data and model its distribution; and this chapter extends representation learning to the hardest and most general temporal data type of all, irregular marked events in continuous time and the graphs they live on. With it, Part IV has covered the three faces of learning temporal structure without a forecasting target: discriminative, generative, and event-based. The next part turns from representing temporal data to quantifying our uncertainty about it.
Prerequisites
This chapter rests on the data-type vocabulary of Part I and the sequence models of Part III. From Chapter 1, Section 1.4 you need the taxonomy of temporal data types, in particular the distinction between a regularly sampled time series and an event sequence or point pattern, because this chapter lives entirely in the latter world and that distinction is the hinge the whole chapter turns on. From Chapter 2: Temporal Data Engineering you need the handling of irregularly sampled data, timestamps, and the leakage discipline of splitting by time, since an event log is the canonical irregular dataset and the survival models here are especially easy to corrupt with future information. From the sequence models of Part III, especially Chapter 10: Recurrent Neural Networks, you need the recurrent and attention mechanisms that the neural temporal point processes and dynamic graph models reuse as their backbone for summarizing event history into a state. A working comfort with conditional probability, the exponential distribution, and hazard rates from the probability appendix will make the intensity and survival mathematics read smoothly. Readers wanting a refresher on any of these will find the relevant chapters and appendices indexed in the Table of Contents.
If you keep one idea from this chapter, keep this: when data arrives as marked events scattered in continuous time, the timing of the events is itself the signal, and you model it by learning a conditional intensity function (how likely an event is to fire right now given the past) rather than by forecasting values on a grid. The Hawkes process makes intensity self-exciting in a fixed parametric form, neural temporal point processes learn that intensity from data, temporal and dynamic graph models learn it for entities embedded in an evolving network, and survival analysis turns the same machinery into time-to-event prediction with proper handling of censored records. Across all of them the unifying object is the intensity (or equivalently the hazard), and the unifying discipline is to respect the arrow of time and never let a future event inform a past prediction.
Chapter Roadmap
- 18.1 Event Streams and Marked Sequences The data type at the heart of the chapter: events stamped with a time and a mark, inter-event times, the counting process and the conditional intensity function that defines them, and why irregular marked events demand a different modeling stance than values on a regular grid.
- 18.2 Temporal Point Processes (Hawkes, neural TPPs) The central model family: the classical self-exciting Hawkes process whose events raise the intensity of future events, its estimation by maximum likelihood, and the neural temporal point processes that replace the fixed intensity form with a learned recurrent or attention-based one, including intensity-free formulations.
- 18.3 Temporal and Dynamic Graphs Events on a network: when the entities form a graph whose edges arrive over time, temporal graph attention (TGAT) and temporal graph networks (TGN) learn node representations whose neighborhoods evolve event by event, unifying the point-process and graph views of interacting entities.
- 18.4 Event Prediction and Time-to-Event Modeling The questions practitioners actually ask: when does the next event occur, and will an event of interest happen at all; survival analysis and the hazard function, the central problem of censoring, the Cox proportional-hazards model, and its deep descendants DeepSurv and DeepHit.
- 18.5 Process Mining From events to processes: discovering the structure of a workflow from its event log, the directly-follows graph and process-discovery algorithms, conformance checking against a model, and next-activity prediction, the bridge from individual events to the business or clinical process that generates them.
Once you have worked through the five sections, the Hands-On Lab below chains them into a single experiment on a real event log. Each lab step maps to a section, so the lab doubles as a review: by the time you finish you will have fit a Hawkes process from Section 18.2, built a censored survival model from Section 18.4, and mined and predicted a process from Section 18.5, all on the marked event streams that Section 18.1 defines.
Hands-On Lab: From Events to Predictions
Objective
Take a single stream of marked events and put every tool of the chapter to work on it, so that by the end you can move from raw timestamps to three different kinds of prediction. You will first fit a self-exciting Hawkes process to an event stream and read off how strongly each event excites the ones that follow, the core temporal-point-process move of Section 18.2, working with the marked event representation of Section 18.1. You will then turn to time-to-event prediction, building a survival model on a cohort in which many records are censored (the event of interest has not happened by the end of observation) and seeing exactly why censoring cannot be ignored or treated as a non-event, the central lesson of Section 18.4. Finally you will treat the same data as a process: mine a directly-follows process model from the event log, check how well it fits, and train a next-activity predictor that says what the entity will do next, the process-mining workflow of Section 18.5. The single thread through all three is that the timing and ordering of events, not their values on a grid, carry the information you are extracting.
What You'll Practice
- Representing data as a marked event stream with timestamps, marks, and inter-event times, following Section 18.1.
- Fitting a Hawkes process by maximum likelihood and interpreting its self-excitation, following Section 18.2.
- Building a time-to-event survival model that correctly handles right-censored records, following Section 18.4.
- Reading a survival curve and a hazard ratio rather than a point prediction, following Section 18.4.
- Discovering a directly-follows process model from an event log and checking its conformance, following Section 18.5.
- Training and evaluating a next-activity predictor on the same log, following Section 18.5.
Setup
You need three small, focused libraries: tick or a comparable point-process package for the Hawkes fit of Section 18.2, lifelines for the survival analysis of Section 18.4, and pm4py for the process mining of Section 18.5. The dataset is a single event log with one row per event, carrying at minimum a case identifier, a timestamp, and an activity or mark; the irregularly sampled clinical record of Chapter 2 works well, as does any public business-process log such as the BPI Challenge logs that ship with pm4py. The one discipline that matters throughout is the leakage discipline of Chapter 2 carried into censored data: when you build the survival cohort you must define the observation window first and mark as censored every record whose event of interest had not occurred by the window's end, never silently dropping it or coding it as a negative, because a dropped or mislabeled censored record biases every estimate that follows. The code below fits a Cox proportional-hazards survival model with explicit censoring in a few lines, the time-to-event core of the lab.
from lifelines import CoxPHFitter # survival model with censoring (Section 18.4)
# df columns: feature columns, plus "duration" (time observed) and
# "event_observed" (1 if the event happened, 0 if right-censored).
cph = CoxPHFitter()
cph.fit(df, duration_col="duration", event_col="event_observed") # censoring handled natively
cph.print_summary() # hazard ratios per feature, not a single point prediction
survival = cph.predict_survival_function(df.iloc[:5]) # full survival curve per subject
Steps
Step 1: Build the marked event stream
Load the event log and shape it into the marked-event representation of Section 18.1: a sorted sequence of (time, mark) pairs per case, with inter-event times computed. Inspect the inter-event time distribution and confirm the events are genuinely irregular, so that grid-based resampling would discard the very structure you are about to model.
Step 2: Fit a Hawkes process and read its self-excitation
Fit a Hawkes process to one event type's stream and recover its baseline intensity and its excitation kernel, the self-exciting model of Section 18.2. Read off the branching ratio: how many follow-on events, on average, each event triggers, and judge whether the stream is bursty and self-exciting or close to a memoryless Poisson process.
Step 3: Build a censored survival model
Define an observation window and an event of interest, then assemble the survival cohort with each record marked as event-observed or right-censored, exactly the censoring discipline of Section 18.4. Fit a Cox proportional-hazards model and read the hazard ratios and survival curves rather than a single time-to-event point estimate.
Step 4: Mine a directly-follows process model
Treat the log as a process and discover a directly-follows graph from it, the process-discovery move of Section 18.5. Inspect the most common paths through the process, then run a conformance check to find the cases that deviate from the discovered model, the variants that a grid view would never surface.
Step 5: Predict the next activity
Train a next-activity predictor on the prefix of each case and evaluate it on held-out cases, the predictive side of Section 18.5. Respecting the leakage discipline of Chapter 2, split by time or by case so that no future event of a case informs a prediction about its own past.
Expected Output
The lab produces three predictions of three different shapes from one event stream, which is the point. The Hawkes fit of Step 2 yields a baseline intensity and an excitation kernel with an interpretable branching ratio, telling you in one number how self-exciting the stream is, and on a clustered finance or alarm stream that ratio is well above zero where a memoryless Poisson model would miss the clustering entirely. The survival model of Step 3 yields hazard ratios and per-subject survival curves computed correctly over a cohort with many censored records, demonstrating concretely why dropping or mislabeling the censored cases would have biased the result. The process model of Step 4 yields a directly-follows graph of the workflow with its main paths and its deviating variants surfaced by conformance checking, and Step 5 yields a next-activity predictor with a held-out accuracy you can read against the trivial most-frequent-next baseline. The reader finishes able to take any marked event log and choose the right tool for the question being asked: a point process when the question is about timing and clustering, a survival model when it is about whether and when an event of interest occurs, and process mining when it is about the structure of the workflow as a whole.
Each movement of this chapter has a mature library that collapses the from-scratch derivation into a handful of calls. The maximum-likelihood Hawkes fit that Section 18.2 derives by hand, log-likelihood, compensator, and optimizer, is a single tick estimator once the events are in shape; the survival model with its careful censored likelihood, which Section 18.4 builds up from the hazard function, is the three lines of lifelines above with censoring handled natively; and the process discovery, conformance checking, and next-activity prediction of Section 18.5 are each one pm4py call over a log dataframe, with the directly-follows mining, the alignment-based conformance, and the encoding all handled internally. The discipline the chapter teaches still governs the libraries: a Hawkes fit on leaked or mis-sorted events, a survival estimate that silently dropped its censored records, or a next-activity model split without respecting the arrow of time will each return a confident and wrong number in exactly the same few lines. The library makes the fit free; it does not make the setup correct.
Stretch Goals
- Replace the parametric Hawkes process of Step 2 with a neural temporal point process from Section 18.2 and compare its held-out log-likelihood against the Hawkes fit, testing whether the learned intensity beats the fixed self-exciting form.
- Lift the event stream onto a graph, treating cases or entities as nodes and shared events as time-stamped edges, and train a temporal graph network from Section 18.3 to predict the next interaction, comparing it against the next-activity predictor of Step 5.
- Swap the Cox model of Step 3 for DeepSurv or DeepHit from Section 18.4 and compare their concordance index against the linear Cox baseline, learning when the deep survival model earns its complexity.
What's Next?
This chapter closes Part IV: Temporal Representation Learning. Across three chapters the part learned to represent temporal data without leaning on a forecasting target: Chapter 16 learned discriminative embeddings of regular series through self-supervised and contrastive objectives, Chapter 17 learned to generate and model the distribution of temporal data, and this chapter learned to represent the most general temporal data type of all, irregular marked events in continuous time and the evolving graphs they live on. Every model in the book so far, whether it forecasts, represents, or generates, has handed you a point: a predicted value, an embedding, a most-likely next event. Part V: Uncertainty, Online, and Adaptive Learning opens with Chapter 19: Probabilistic Forecasting and Uncertainty Quantification, which insists that a temporal prediction without a calibrated measure of its own uncertainty is only half a prediction, and builds the probabilistic and conformal machinery to attach honest intervals to every forecast. The representations you learned to build in Part IV become, in Part V, representations you learn to trust exactly as much as they deserve.
Bibliography & Further Reading
Point Processes: Foundations
Hawkes, A. G. "Spectra of Some Self-Exciting and Mutually Exciting Point Processes." Biometrika, 58(1), 83-90, 1971. doi.org/10.1093/biomet/58.1.83
The original paper introducing the self-exciting point process that now bears its name, the classical foundation of the temporal point processes in Section 18.2 and the model fit in the lab.
Daley, D. J., Vere-Jones, D. "An Introduction to the Theory of Point Processes, Volumes I and II." Springer, 2nd edition, 2003-2008. link.springer.com/book/10.1007/b97277
The definitive reference on point-process theory, the conditional intensity, and the counting process, the rigorous backing for the formalism of Section 18.1.
Neural Temporal Point Processes
Du, N., Dai, H., Trivedi, R., Upadhyay, U., Gomez-Rodriguez, M., Song, L. "Recurrent Marked Temporal Point Processes: Embedding Event History to Vector." KDD, 2016. kdd.org/kdd2016/papers/files/rpp1081-duA.pdf
RMTPP, the recurrent marked temporal point process that first embedded event history into a recurrent state to learn the intensity, the opening neural model of Section 18.2.
Mei, H., Eisner, J. "The Neural Hawkes Process: A Neurally Self-Modulating Multivariate Point Process." NeurIPS, 2017. arXiv:1612.09328. arxiv.org/abs/1612.09328
The neural Hawkes process, a continuous-time LSTM whose intensity is self-modulating, the neural generalization of the classical Hawkes model in Section 18.2.
Zuo, S., Jiang, H., Li, Z., Zhao, T., Zha, H. "Transformer Hawkes Process." ICML, 2020. arXiv:2002.09291. arxiv.org/abs/2002.09291
The Transformer Hawkes process, replacing recurrence with self-attention over event history to model long-range dependencies between events, the attention-based TPP of Section 18.2.
Shchur, O., Bilos, M., Gunnemann, S. "Intensity-Free Learning of Temporal Point Processes." ICLR, 2020. arXiv:1909.12127. arxiv.org/abs/1909.12127
An intensity-free formulation that models the inter-event time distribution directly with a normalizing flow or mixture, sidestepping the intractable intensity integral, the intensity-free strand of Section 18.2.
Temporal and Dynamic Graphs
Xu, D., Ruan, C., Korpeoglu, E., Kumar, S., Achan, K. "Inductive Representation Learning on Temporal Graphs." ICLR, 2020. arXiv:2002.07962. arxiv.org/abs/2002.07962
TGAT, temporal graph attention with a functional time encoding for inductive learning on graphs whose edges arrive over time, the attention-based dynamic graph model of Section 18.3.
Rossi, E., Chamberlain, B., Frasca, F., Eynard, D., Monti, F., Bronstein, M. "Temporal Graph Networks for Deep Learning on Dynamic Graphs." ICML Workshop on Graph Representation Learning, 2020. arXiv:2006.10637. arxiv.org/abs/2006.10637
TGN, temporal graph networks combining memory modules with graph attention to learn node representations on continuous-time dynamic graphs event by event, the central model of Section 18.3 and a lab stretch goal.
Survival and Time-to-Event Analysis
Cox, D. R. "Regression Models and Life-Tables." Journal of the Royal Statistical Society: Series B, 34(2), 187-220, 1972. doi.org/10.1111/j.2517-6161.1972.tb00899.x
The Cox proportional-hazards model, the foundational regression for censored time-to-event data, the classical survival model of Section 18.4 and the lab's baseline.
Katzman, J. L., Shaham, U., Cloninger, A., Bates, J., Jiang, T., Kluger, Y. "DeepSurv: Personalized Treatment Recommender System Using a Cox Proportional Hazards Deep Neural Network." BMC Medical Research Methodology, 2018. arXiv:1606.00931. arxiv.org/abs/1606.00931
DeepSurv, a neural network parameterizing the Cox partial likelihood for nonlinear survival prediction, the deep descendant of Cox regression in Section 18.4.
Lee, C., Zame, W. R., Yoon, J., van der Schaar, M. "DeepHit: A Deep Learning Approach to Survival Analysis with Competing Risks." AAAI, 2018. ojs.aaai.org/index.php/AAAI/article/view/11842
DeepHit, modeling the survival distribution directly without a proportional-hazards assumption and handling competing risks, the second deep survival model of Section 18.4 and a lab stretch goal.
Process Mining
van der Aalst, W. M. P. "Process Mining: Data Science in Action." Springer, 2nd edition, 2016. link.springer.com/book/10.1007/978-3-662-49851-4
The standard textbook on process mining, covering process discovery, conformance checking, and enhancement from event logs, the foundation of Section 18.5.
Tools and Libraries
Davidson-Pilon, C. "lifelines: Survival Analysis in Python." lifelines.readthedocs.io
The reference Python library for survival analysis, supplying Cox regression, Kaplan-Meier estimation, and native censoring in a few lines, the engine behind the time-to-event step of the lab.
Berti, A., van Zelst, S. J., van der Aalst, W. M. P. "pm4py: Process Mining for Python." Fraunhofer FIT. pm4py.fit.fraunhofer.de
The reference Python library for process mining, providing directly-follows discovery, conformance checking, and next-activity prediction over event logs, the engine behind the process-mining step of the lab.