Time Stamps, Time Zones, and Event Ordering

"I am only a number. I count seconds since one particular midnight in 1970 and I count them in one place only. Everything you believe about my afternoon, my Tuesday, my daylight saving, you added yourself, and you may have added it wrong."

A Timestamp That Refuses to Localize

This is the first section of Chapter 2, and it deliberately starts beneath the data. The previous chapter argued that time changes the modeling problem qualitatively; this chapter is about the unglamorous engineering that has to be correct before any of that modeling is even meaningful. We begin with the timestamp itself because it is the atom of every later operation. The five subsections move from the physical (what a timestamp is) through the human (time zones and daylight saving), through the architectural (event time versus processing time), into the structural (ordering and monotonicity), and finally to the conceptual (calendars and clocks that do not tick in seconds). Two runnable examples anchor the two most expensive mistakes: a daylight saving bug and a non-monotonic index. The unified notation for time indices used across the book is fixed in Appendix A.

1. What a Timestamp Really Is Beginner

Strip away the formatting and a timestamp is an integer: a count of elapsed time units since a fixed reference instant called the epoch. The near-universal convention, Unix or POSIX time, counts seconds (or, at finer resolution, milliseconds, microseconds, or nanoseconds) since midnight UTC on 1 January 1970. This is the epoch-time view, and it has a decisive property: it is a single scalar on one monotone axis, with no time zone, no calendar, and no ambiguity. The string "2026-03-14 09:30:00" that a human reads is the wall-clock view, and it is not a number at all until you also say where on Earth that wall clock hangs, because the same instant reads as a different wall-clock string in every zone. Epoch time is what machines should compute on; wall-clock strings are what humans should see, and only at the edge.

Resolution is the second thing to pin down. NumPy and pandas represent instants with the datetime64 type, a 64-bit integer paired with a unit. At nanosecond resolution that integer range spans roughly the years 1678 to 2262, which is ample for most data but a real boundary for geological or astronomical series, where coarser units are used instead. A duration, the difference of two instants, is a timedelta64, the same integer-with-unit idea. Because these are integers under the hood, arithmetic on them is exact and fast, which is exactly what we want for indexing millions of rows.

A third distinction matters the moment you measure elapsed time rather than label an instant: the difference between a wall clock and a monotonic clock. A wall clock reports civil time and can jump, forward at a daylight saving spring transition, backward when an administrator or NTP daemon corrects it, so subtracting two wall-clock readings can yield a negative or absurd duration. A monotonic clock only ever moves forward and is the correct source for "how long did this take", though it carries no calendar meaning and cannot be compared across machines or reboots. Use the wall clock to say when; use a monotonic clock to measure how long. Code 2.1.1 makes the epoch-versus-wall-clock duality concrete and shows the two clock sources side by side.

import time
from datetime import datetime, timezone

# Wall-clock instant for one event, stored the only safe way: as UTC.
instant_utc = datetime(2026, 3, 14, 9, 30, 0, tzinfo=timezone.utc)

# The same instant as a single epoch scalar: seconds since 1970-01-01 UTC.
epoch_seconds = instant_utc.timestamp()
print("epoch seconds:", epoch_seconds)                 # one number, no time zone
print("round-trips to:", datetime.fromtimestamp(epoch_seconds, tz=timezone.utc))

# Two clock sources. time.time() is the WALL clock and can jump.
# time.monotonic() never goes backward and is the right tool for durations.
t0_wall = time.time()
t0_mono = time.monotonic()
total = sum(i * i for i in range(2_000_000))           # some work to time
print("wall delta  :", time.time() - t0_wall, "s  (can be wrong if clock jumps)")
print("mono delta  :", time.monotonic() - t0_mono, "s  (always >= 0)")

epoch seconds: 1773480600.0
round-trips to: 2026-03-14 09:30:00+00:00
wall delta  : 0.0712 s  (can be wrong if clock jumps)
mono delta  : 0.0711 s  (always >= 0)

2. Time Zones and the Daylight Saving Trap Intermediate

A datetime is naive when it carries no zone information and aware when it knows its offset from UTC. Naive datetimes are landmines: "2026-11-01 01:30:00" with no zone is genuinely ambiguous, and any operation that assumes otherwise is guessing. The discipline that removes the ambiguity is simple to state and surprisingly hard to follow under deadline pressure: treat UTC as the single ground truth, store every instant in UTC, and convert to a local zone only at the very edge where a human reads it or a calendar rule applies. Localize on input, compute in UTC, localize on output. Figure 2.1 dramatizes why a single canonical clock is non-negotiable.

Daylight saving transitions are where naive handling goes from sloppy to wrong. Twice a year, in zones that observe it, the local wall clock does something a monotone axis never does. In autumn it falls back, so a local hour repeats: 01:30 local time occurs twice, once before and once after the transition, and a bare local string cannot tell them apart. In spring it springs forward, so a local hour vanishes: 02:30 simply never exists on that date. Resampling, joining, or differencing on naive local timestamps across such a boundary silently produces duplicated rows, missing rows, or hour-long errors in every elapsed-time calculation. Code 2.1.2 reproduces the autumn duplicated-hour bug on a real zone and then fixes it with the UTC-first discipline.

import pandas as pd
from zoneinfo import ZoneInfo

# Six hourly readings straddling the US autumn DST fall-back (2 a.m. -> 1 a.m.)
# on 2 November 2025. Written as NAIVE local strings, the way logs often arrive.
naive_local = pd.to_datetime([
    "2025-11-02 00:30", "2025-11-02 01:30", "2025-11-02 01:30",
    "2025-11-02 02:30", "2025-11-02 03:30", "2025-11-02 04:30",
])
readings = pd.Series([10, 11, 12, 13, 14, 15], index=naive_local)

# WRONG: assume these naive strings are already UTC and just diff them.
naive_gaps = readings.index.to_series().diff()
print("naive hour gaps:\n", naive_gaps.dt.total_seconds().div(3600).tolist())

# RIGHT: localize to the real zone (resolving the repeated hour), then
# convert to UTC, which is strictly monotone, before any time arithmetic.
ny = ZoneInfo("America/New_York")
aware_local = readings.index.tz_localize(
    ny, ambiguous=[False, False, True, False, False, False]  # 2nd 01:30 is post-fallback
)
utc_index = aware_local.tz_convert("UTC")
utc_gaps = utc_index.to_series().diff().dt.total_seconds().div(3600)
print("utc   hour gaps:\n", utc_gaps.tolist())

naive hour gaps:
 [nan, 1.0, 0.0, 1.0, 1.0, 1.0]
utc   hour gaps:
 [nan, 1.0, 1.0, 1.0, 1.0, 1.0]

The ambiguous argument in Code 2.1.2 is doing real work: it tells pandas which side of the fall-back each repeated wall-clock time belongs to. The matching nonexistent argument handles the spring spring-forward, where a local time has no UTC counterpart and you must say whether to shift forward, roll back, or raise. The deeper lesson is that you should almost never localize ambiguously by accident. Keep data in UTC from ingestion onward and these arguments rarely surface, because the only place a repeated or missing hour can bite you is the boundary conversion itself.

3. Event Time Versus Processing Time Intermediate

Once instants are stored correctly, a subtler distinction governs how a temporal system behaves: an event has two timestamps, not one. The event time is when the thing actually happened in the world (a trade executed, a sensor reading was taken, a click occurred). The processing time (also called ingestion or arrival time) is when your system observed and recorded it. In a perfect synchronous world these coincide, but networks buffer, devices go offline and replay their backlog, batch jobs land late, and clocks drift, so in practice the two diverge. Write the event time of observation $i$ as $\tau_i$ and its processing time as $\rho_i$; the per-event delay is

and the whole discipline of streaming temporal systems is built around the fact that $\delta_i$ is positive, variable, and occasionally enormous. When events are sorted by event time $\tau$ they form the clean sequence the model wants; when they arrive sorted by processing time $\rho$ they can be badly out of order, because a high-delay event with a small $\tau$ can land after a low-delay event with a larger $\tau$.

Two consequences follow directly. A late event is one whose processing time falls after a window over its event time has already been computed and emitted, so the window's result was wrong and must either be revised or discarded. An out-of-order event is one that arrives after an event with a later event time, breaking the assumption that arrival order equals event order. Both are normal, not pathological, in any real stream. The leakage discipline of Section 2.6 sharpens the stakes further: a feature for the prediction at event time $t$ may use only information whose processing time is at or before $t$, because that is all a live system would actually have. Confusing "everything with event time before $t$" for "everything available by processing time $t$" is one of the most common ways a backtest leaks the future.

4. Ordering and Monotonicity Intermediate

Almost every temporal operation, rolling windows, resampling, asof joins, differencing, lag features, assumes the index is sorted in increasing time and has no duplicates. Make that assumption explicit. A well-formed temporal index is a strictly increasing sequence

with no ties (which would be duplicates) and known, deliberate spacing (regular if $t_{i+1} - t_i$ is constant, irregular otherwise, but in either case understood rather than accidental). Three properties are worth checking on every dataset before modeling: monotonicity (is it sorted), uniqueness (are there duplicate timestamps), and gap structure (where are the holes, and are they expected). Pandas exposes the first two as cheap flags, index.is_monotonic_increasing and index.is_unique, and reconstructs the third with a diff on the index.

The reason to be strict is that a non-monotonic or duplicated index does not raise; it silently produces wrong answers. A rolling mean over an unsorted index mixes the future into the past within each window. A duplicate timestamp double-counts in any aggregation and can make a merge_asof match the wrong row. A resample over an index with an unnoticed multi-day gap fabricates rows by forward-filling across the hole, manufacturing data that never existed. Because windowing is the foundation of nearly every feature in Chapter 2, a corrupt index poisons the feature matrix before any model sees it. The validator in Code 2.1.3 makes the three properties auditable in one pass and is the kind of guard worth running at every ingestion boundary.

import pandas as pd
import numpy as np

def audit_temporal_index(idx: pd.DatetimeIndex, expected_freq="h"):
    """Return a dict of health checks for a temporal index, plus the gap list."""
    sorted_idx = idx.sort_values()
    step = pd.Timedelta(1, unit=expected_freq)
    gaps = sorted_idx.to_series().diff()
    return {
        "is_monotonic": bool(idx.is_monotonic_increasing),  # already sorted?
        "is_unique":    bool(idx.is_unique),                 # no duplicate stamps?
        "n_duplicates": int(idx.duplicated().sum()),
        "n_missing_steps": int(((gaps - step) / step).round().clip(lower=0).sum()),
        "max_gap": gaps.max(),                               # largest hole
    }

# A deliberately broken index: out of order, one duplicate, and a 3-hour hole.
broken = pd.to_datetime([
    "2026-01-01 00:00", "2026-01-01 02:00", "2026-01-01 01:00",  # 02:00 before 01:00
    "2026-01-01 01:00",                                          # duplicate 01:00
    "2026-01-01 06:00",                                          # jumps past 03,04,05
])
report = audit_temporal_index(pd.DatetimeIndex(broken))
for k, v in report.items():
    print(f"{k:>16}: {v}")

# The repair recipe: sort, drop exact-duplicate timestamps, then proceed.
clean = pd.DatetimeIndex(broken).sort_values()
clean = clean[~clean.duplicated(keep="first")]
print("clean & monotonic:", clean.is_monotonic_increasing, "| unique:", clean.is_unique)

    is_monotonic: False
       is_unique: False
    n_duplicates: 1
 n_missing_steps: 4
         max_gap: 0 days 05:00:00
clean & monotonic: True | unique: True

5. Calendars, Business Time, and Irregular Clocks Advanced

The final idea loosens the assumption that "time" means seconds at all. Many series live on a business calendar rather than a physical one. A stock index has no value at 3 a.m. on a Sunday, not because the data is missing but because the market is closed; a daily-returns model that naively assumes 365 evenly spaced days will compute a wrong overnight gap across every weekend and holiday. Trading calendars encode exactly which days and hours are valid, and libraries such as pandas business-day offsets and the exchange_calendars project turn "the next valid trading day" into a single call rather than a hand-maintained holiday list that goes stale.

Pushing the idea further, the natural clock for some processes is an event counter, not a wall clock. Sampling "every 100 trades" (volume or tick bars) instead of "every minute" produces bars with far more uniform statistical properties, because activity, not the rotation of the Earth, drives the relevant dynamics. In the running healthcare dataset of this book, a patient's vitals are recorded when clinically warranted, so the meaningful index is the sequence of measurements, not a regular grid; resampling it to fixed minutes would both invent data and erase the information carried by when a measurement was taken. The reframing is liberating: time is whatever monotone index best parameterizes the process, and sometimes it counts events, not seconds. This perspective returns when we handle irregular sampling architecturally in Chapter 13 and when we model event sequences directly in Chapter 18.

Taken together, the five subsections form a single contract for the rest of Chapter 2. Know that an instant is one scalar (subsection 1); keep that scalar in UTC and localize only at the edge (subsection 2); never confuse when an event happened with when you saw it (subsection 3); guarantee a sorted, unique, gap-aware index before any window (subsection 4); and remember that the right time axis is sometimes a calendar or an event counter rather than seconds (subsection 5). With this foundation in place, Section 2.2 can resample and aggregate safely, because the index it operates on is finally trustworthy.