"Ask me to forecast next quarter's demand and I will not improvise a number from the shimmer of my weights. I will reach for the calculator, I will reach for the forecaster you trained in Part III, I will check whether it is confident enough to be trusted, and only then will I speak. I would rather call a tool than guess; guessing is how language models embarrass themselves at meetings."
An Agent That Would Rather Call a Calculator Than Guess
An agent is only as useful as the gap between what it can plan and what it can actually do, and tools are how that gap is closed. Planning decomposes an open-ended goal into an ordered sequence of executable steps; tool use lets each step reach outside the model's frozen weights to a calculator, a search index, a database, or, the move that makes this a temporal-AI book and not a generic agents chapter, to the forecasters and causal estimators you built in the preceding thirty chapters. This section traces the arc from classical planning (state-space search and PDDL, where the action space is small and known) to LLM planning (chain-of-thought, tree-of-thought, and plan-and-execute, where the action space is open-ended natural language), shows how ReAct interleaves a reasoning trace with structured function calls so the agent observes the world between thoughts, and then makes the temporal payoff concrete: an agent that calls a trained forecaster as a tool, reads its calibrated prediction interval from Chapter 19, and refuses to trust a forecast whose uncertainty is too wide. It closes with the reliability machinery (tool-error handling, output verification, and self-correcting reflection loops) and a worked example that builds a plan-and-execute agent from scratch around a forecasting tool, then collapses it to a few lines with a library tool-calling agent. You leave able to give a temporal agent a multi-step goal and have it earn its answer by calling the right model at the right time.
In Section 31.2 we gave the agent a memory: a way to carry temporal context across turns so it could reason about what had already happened. Memory tells the agent what is true; this section tells it what to do about it. An agent with perfect recall and no plan is a historian, not an actor. The competency we install here is the bridge from situation to action: take a goal stated in open-ended language ("tell me whether to pre-order inventory for the holiday spike, and by how much"), break it into steps the agent can execute, and equip each step with a tool that does the part the language model itself cannot do reliably, namely arithmetic, retrieval, and quantitative forecasting. We use the unified notation of Appendix A where it helps: a goal $g$, a plan $\pi = (a_1, a_2, \dots, a_n)$ as an ordered sequence of actions, a tool invocation as an action $a_i$ that returns an observation $o_i$, and the running context $c_t$ the agent conditions on at step $t$.
Why does planning deserve its own treatment rather than "just prompt the model to think step by step"? Because the planning problem here is genuinely harder than the one Part VI solved. The Monte Carlo Tree Search of Chapter 29 and the model predictive control of Chapter 27 plan beautifully, but they plan over a closed, enumerable action space against a known transition model. A temporal agent plans over an open-ended action space (any tool call, any natural-language sub-goal, any query it can phrase) with no clean transition model and a horizon that is not fixed in advance. That open-endedness is what forces the shift from the search-based planners of classical AI to the language-model planners of this section, and it is why a tool call, the act of stepping outside the model to get a real observation, becomes the load-bearing primitive: it is how an agent in an unmodeled world gets the ground truth that MCTS gets for free from its simulator.
The four concrete competencies this section installs are these: to decompose a goal into a plan and to know when classical search applies versus when an LLM planner is the only option; to wire tool use through the ReAct loop with structured tool schemas so reasoning and acting interleave; to register this book's own models (a forecaster from Chapter 14, a causal estimator from Chapter 30) as agent tools and to gate their use on the calibrated uncertainty of Chapter 19; and to make the resulting agent reliable through verification and reflection. These are the skills that turn a chat model into a temporal system that acts.
1. Planning in Agents: From a Goal to a Sequence of Steps Beginner
Planning is the act of turning a goal into an ordered sequence of steps whose execution achieves it. Formally, given a goal $g$ and a set of available actions $\mathcal{A}$, a planner produces a plan $\pi = (a_1, \dots, a_n)$ with each $a_i \in \mathcal{A}$, such that executing the actions in order transforms the initial state into one that satisfies $g$. Everything interesting about agent planning is hidden in two questions: how large and how known is $\mathcal{A}$, and how reliably can the planner predict what each action does. The answers separate the two traditions we now contrast.
The classical tradition treats planning as search. In the state-space view, you have an explicit set of states, a set of operators with preconditions and effects, and you search (with A*, breadth-first, or heuristic search) for a path from the start state to a goal state. The Planning Domain Definition Language (PDDL) formalizes this: a domain file declares predicates and actions with typed preconditions and effects, a problem file declares the initial state and goal, and a solver returns a provably correct action sequence. Classical planning is exact, verifiable, and complete when the domain is fully specified, and it is exactly the right tool when the world is small and known: a logistics routing problem, a manufacturing job-shop schedule, a robot stacking blocks. Its fatal limitation for open-ended agents is that someone must write down every predicate and every action effect in advance, and the real tasks we want temporal agents to do ("analyze this sensor anomaly and recommend a maintenance action") do not come with a PDDL domain.
The LLM tradition treats planning as generation. Instead of searching an enumerated space, a language model generates the plan as text, conditioned on the goal and a description of the available tools. The simplest form is chain-of-thought: prompt the model to "think step by step" and it emits a linear reasoning trace that doubles as a plan. Tree-of-thought generalizes this by exploring several reasoning branches and selecting among them, recovering a shadow of classical search but over natural-language thoughts rather than enumerated states. The pattern most used in production agents is plan-and-execute: a planner module first writes the whole plan as an explicit list of steps, then an executor module carries the steps out one at a time, optionally re-planning when a step fails. The strength is generality (no domain file, any goal expressible in language); the weakness is that the plan is only as sound as the model's implicit world knowledge, with no completeness guarantee and a real risk of hallucinated steps.
| Dimension | Classical planning (search / PDDL) | LLM planning (CoT / ToT / plan-and-execute) |
|---|---|---|
| action space $\mathcal{A}$ | small, enumerated, declared in advance | open-ended natural language and tool calls |
| transition model | explicit operator preconditions and effects | implicit in the model's weights, unverified |
| how a plan is produced | systematic search (A*, heuristic search) | generated as text, optionally branched |
| guarantees | sound and complete when domain is correct | none; may hallucinate steps or tools |
| handles novelty | poorly; needs a hand-written domain | well; generalizes from language |
| best when | world is small and fully known | world is open and messy |
The connection to Part VI is direct and worth making explicit, because it explains both what LLM planning borrows and what it gives up. The MCTS of Chapter 29 and the receding-horizon MPC of Chapter 27 are planners in exactly this sense: they decompose a goal (maximize return) into a sequence of actions by looking ahead through a model of the world. Tree-of-thought is recognizably MCTS with the simulator replaced by a language model's judgment and the enumerated action set replaced by free-form thoughts. What the agent gains is the ability to plan in a world no one wrote a transition model for; what it loses is the simulator that let MCTS evaluate a leaf by rolling it out. That missing simulator is the whole motivation for tool use: when the agent cannot simulate the consequence of a step, it executes the step against a tool and observes the real result. Planning and tool use are therefore two halves of one mechanism, the open-world replacement for search-with-a-simulator.
The single fact that separates agent planning from the planning of Part VI is the openness of the action space. MCTS and MPC plan over a closed, enumerable set of actions against a known transition model, and that closedness is precisely what lets them search and guarantee. An agent in the real world faces an action space that is effectively infinite (any tool call, any sub-goal phrasable in language) and a transition model it does not have. You cannot enumerate what you cannot list, and you cannot search what you cannot enumerate, so the agent generates a plan instead of searching for one, and substitutes a real tool call for the simulator rollout it lacks. Every design choice in the rest of this section follows from this one constraint: no closed action space, no simulator, so generate-and-act replaces search-and-simulate.
2. Tool Use and Function Calling: Reaching Outside the Weights Beginner
A language model's weights are frozen at training time, which makes the model bad at three things it is constantly asked to do: exact arithmetic, retrieval of facts it never saw or that changed after training, and any computation that needs an external system (a database query, a live forecast, a code execution). Tool use fixes this by giving the model a set of external functions it can call. A tool is just a function with a name, a documented purpose, a typed input schema, and a return value; the agent decides when to call which tool, emits a structured call, an external runtime executes it, and the result comes back as an observation the agent conditions on next.
The mechanism that makes this reliable is the structured tool schema. Rather than hoping the model writes a callable string, modern function-calling interfaces give the model a machine-readable description of each tool (typically JSON Schema: the tool's name, a natural-language description, and the typed parameters with their constraints) and the model emits a structured object naming the tool and its arguments. The runtime validates that object against the schema before executing, so a malformed call is caught rather than run. This is the same discipline that makes classical planning verifiable, applied to one action at a time: the schema is the precondition, the validated call is the operator, the returned observation is the effect.
The dominant pattern for weaving tool calls into reasoning is ReAct (Reason + Act): the agent alternates between a thought (a private reasoning step in natural language) and an action (a tool call), and after each action it receives an observation (the tool's result) before thinking again. The loop is thought, action, observation, thought, action, observation, ... until the agent emits a final answer. The reason interleaving matters, rather than planning the whole sequence up front, is that each observation can change the plan: an agent that searches, reads the result, and then decides what to compute is strictly more capable than one that commits to all its steps blind. ReAct is plan-and-execute compressed to a horizon of one, re-planning after every observation, and it is the workhorse loop of practical agents.
The space of useful tools is wide, and the temporal-AI agent draws on a particular slice of it. Generic tools include a calculator (exact arithmetic), a search or retrieval tool (current facts, document lookup), a code-execution sandbox (arbitrary computation, plotting), and a database query tool (structured data). The tools that make this book's agents temporal are the ones we add in the next subsection: a forecasting tool that calls a trained model from Part III, and a causal-estimator tool that calls the methods of Chapter 30. The discipline is identical across all of them: a name, a typed schema, a deterministic runtime, and a returned observation. What differs is only what the function does inside.
There is a quiet comedy in the fact that a model with hundreds of billions of parameters, trained on a meaningful fraction of everything humans have written, cannot reliably multiply two four-digit numbers, and that the fix is to hand it the same calculator a ten-year-old uses. The lesson is not that the model is dumb; it is that the model is a brilliant generalist who should stop pretending to be an arithmetic unit. A good agent knows the boundary of its own competence and delegates across it without ego. The epigraph's agent has internalized exactly this: it would rather call a tool than guess, because it has watched what happens at meetings to language models that guess.
3. Using This Book's Models as Tools: The Temporal-Agent Payoff Intermediate
Here is where a temporal-AI agent stops being a generic chatbot with a calculator and becomes the capstone of this book. The thirty chapters before this one built quantitative temporal models: deep forecasters in Chapter 14, probabilistic and conformal forecasters in Chapter 19, causal estimators in Chapter 30. Each of these is exactly a function with typed inputs and a return value, which is to say each is exactly a tool. Registering a trained forecaster as an agent tool lets the agent answer "what will demand be next quarter" not by improvising a plausible-sounding number from its weights, but by calling a model that was fit to the data and validated on a held-out window.
The deciding ingredient, the one that makes a temporal agent trustworthy rather than merely confident, is calibrated uncertainty gating. A forecast is not a number; it is a number with a quantified uncertainty. The conformal and probabilistic methods of Chapter 19 attach to every prediction a calibrated interval, $[\hat{y} - \delta, \hat{y} + \delta]$ at a stated coverage level (say 90 percent), with the guarantee that the true value falls inside it about 90 percent of the time. The agent reads that interval and uses it as a gate: if the interval is narrow relative to the decision threshold, the forecast is trustworthy and the agent acts on it; if the interval is too wide, the forecast carries too little information, and the agent declines to act on it, instead gathering more data, falling back to a conservative default, or escalating to a human. Formally, the agent trusts a forecast when its relative half-width satisfies
$$\frac{\delta}{|\hat{y}|} \le \tau,$$for a task-specific tolerance $\tau$, and treats the forecast as uninformative otherwise. This single inequality is the temporal-agent payoff: it is how the uncertainty quantification of Chapter 19 becomes an action gate rather than a footnote on a chart. A generic agent reports a forecast; a temporal agent reports a forecast it has checked it is allowed to trust.
An inventory agent must decide whether to pre-order stock for a holiday spike. Its decision rule: pre-order only if forecast demand exceeds the reorder threshold of $1000$ units with enough confidence. Two forecasts arrive from the Chapter 19 conformal forecaster at 90 percent coverage. Forecast A: point estimate $\hat{y}_A = 1400$, interval $[1320, 1480]$, so half-width $\delta_A = 80$ and relative half-width $\delta_A / \hat{y}_A = 80/1400 \approx 0.057$. With tolerance $\tau = 0.15$, this passes the gate ($0.057 \le 0.15$), and since even the interval's lower bound $1320$ clears the $1000$ threshold, the agent pre-orders. Forecast B: point estimate $\hat{y}_B = 1100$, interval $[600, 1600]$, so $\delta_B = 500$ and relative half-width $500/1100 \approx 0.45$. This fails the gate ($0.45 > 0.15$), and indeed the interval straddles the $1000$ threshold ($600 < 1000 < 1600$), so the point estimate of $1100$ is not actionable: the agent declines to commit, requests an updated forecast with more recent data, and escalates. A generic agent would have pre-ordered on B's $1100$ and been wrong as often as the wide interval warned. The gate turned a calibrated interval into a refusal to guess.
The same pattern extends to the causal-estimator tool. Suppose the agent is asked not "what will demand be" but "what would happen to demand if we raised the price by five percent": a causal, interventional question of the kind Chapter 30 answers. The agent calls a causal-estimator tool that returns an estimated treatment effect with a confidence interval, and gates on that interval exactly as it gated the forecast. The discipline composes: a plan may call the forecaster to predict a baseline, the causal estimator to predict the effect of an intervention on that baseline, and the calculator to combine them, with every quantitative step gated on its own uncertainty before the agent builds the next step on top of it. This is the architecture of a temporal agent that decides, not merely describes.
The recurring thread of this book is that each classical idea returns in learned form; the thread of this chapter is that each learned model returns as a tool. The ARIMA of Chapter 5, the deep forecaster of Chapter 14, the conformal predictor of Chapter 19, the causal estimator of Chapter 30: each is a function with a typed input, a return value, and (crucially) a quantified uncertainty, which is exactly the contract a tool needs. The agent of this chapter is the integration layer that the whole book was building toward: it plans over these tools, calls them when a quantitative answer is needed, and gates each call on the calibrated uncertainty that the corresponding chapter taught us to compute. The temporal agent is not a new model; it is the orchestration of every model that came before.
4. Reliability: Tool Errors, Verification, and Reflection Intermediate
An agent that calls tools inherits every way a tool can fail, and a plan that chains tools compounds those failures. The reliability of a temporal agent is therefore not an afterthought but a first-class design concern with three layers: handling the errors a tool returns, verifying the outputs a tool produces, and correcting the agent's own mistakes through reflection.
The first layer is tool-error handling. A tool call can fail in mundane ways (a timeout, a malformed argument the schema validator rejects, a forecasting model that throws because the input window was too short) and the agent must treat these as observations to reason about, not as crashes. A robust agent wraps each call so a failure returns a structured error observation ("forecast failed: insufficient history, need 30 steps, got 12"), which the agent reads and responds to by repairing the call, switching to a fallback tool, or reporting the limitation. The schema validation of subsection two is the cheap front line here: a malformed call is caught before it runs, so most argument errors never reach the tool at all.
The second layer is verification. Because an LLM plan carries no soundness guarantee, the agent should check tool outputs against sanity constraints before building on them: a negative forecast for a strictly-positive demand series, a treatment effect with the wrong sign, a probability outside $[0, 1]$, an interval that does not contain its own point estimate. These checks are cheap, catch a large fraction of silent errors, and are exactly the open-world analogue of the precondition checking that makes classical planning sound. The uncertainty gate of subsection three is itself a verification step: a forecast whose interval is too wide fails verification and is not used.
The third layer is self-correction through reflection. When a step fails or a verification check trips, a reflection loop has the agent examine its own trajectory, diagnose what went wrong, and revise the plan: this is the Reflexion pattern, where the agent generates a natural-language critique of its failed attempt and conditions its retry on that critique. Reflection turns a one-shot failure into an iterative repair, and combined with re-planning (the executor asking the planner for a new plan when the old one stalls) it is what lets an agent recover from the inevitable surprises of an open world rather than confidently proceeding from a broken step.
Four failure modes recur when readers first build tool-using agents, and naming them now saves a debugging afternoon:
- Trusting a forecast without reading its interval. The single most common temporal-agent bug: the agent acts on the point estimate $\hat{y}$ and ignores the calibrated interval, so it commits confidently to a forecast the model itself flagged as uncertain. Always gate on $\delta/|\hat{y}| \le \tau$ as in subsection three.
- No structured schema, so the model writes uncallable strings. Without a validated JSON schema the model emits a tool call the runtime cannot parse, and the failure surfaces as a confusing crash rather than a caught argument error. Validate every call against its schema before executing.
- Treating a tool error as a crash instead of an observation. An agent that aborts on the first tool exception cannot recover; one that reads the error as an observation can repair the call or fall back. Wrap every tool call so failures return structured error observations.
- Unbounded loops. A ReAct loop with no step cap and no progress check can call tools forever when it gets stuck. Always bound the loop and detect non-progress (the same action and observation repeating), as the worked example does.
Agent planning and tool use are among the most active areas in applied AI right now. The ReAct pattern (Yao et al., 2023) and Reflexion (Shinn et al., 2023) set the template, and the 2024 to 2026 frontier pushes it in several directions. The Model Context Protocol (MCP, Anthropic, 2024) standardized how agents discover and call external tools, turning ad-hoc function calling into a portable interface so a forecaster registered once is callable by any compliant agent. Tree-of-thought (Yao et al., 2023) and the language-agent-tree-search line graft Monte Carlo Tree Search onto LLM planning, reconnecting agent planning to the MCTS of Chapter 29. On the temporal side specifically, time-series agents that wrap foundation forecasters such as TimesFM, Moirai, and Chronos (the models of Chapter 15) as callable tools are an emerging 2025 pattern, and pairing them with conformal uncertainty gating is exactly the calibrated-tool-use direction this section teaches. The open problem the field is circling: how to give an open-ended LLM planner the soundness guarantees that classical PDDL planning had, through verifier-in-the-loop and formal-checking hybrids. The practitioner's takeaway for 2026: plan with an LLM, act through validated tools, gate quantitative tools on calibrated uncertainty, and verify before you commit.
5. Worked Example: A Plan-and-Execute Agent That Calls a Forecasting Tool Advanced
We now make the whole section executable. The task is deliberately multi-step and quantitative, the kind a language model cannot answer by improvisation: "Forecast next month's demand from the recent history, and if the 90 percent interval clears the reorder threshold of 1000 units, report how many units to pre-order above the threshold." Answering it requires a forecast (a tool), an uncertainty gate (subsection three), and an arithmetic step (a tool), planned and executed in order. Code 31.3.1 builds the two tools and the planner from scratch, in plain Python with no agent framework, so every mechanism of the section is visible.
import numpy as np
# --- Two tools: each is a function with a typed contract and a return value. ---
def forecast_tool(history, horizon=1, coverage=0.90):
"""Forecasting tool (stands in for a trained Chapter-14 model + Chapter-19
conformal interval). Returns a point estimate and a CALIBRATED interval."""
history = np.asarray(history, dtype=float)
if len(history) < 6: # tool-error handling: precondition check
return {"error": "insufficient history: need >= 6 points, got %d" % len(history)}
# Simple trend+level point forecast (a real model would replace this line).
level = history[-3:].mean()
trend = np.polyfit(np.arange(len(history)), history, 1)[0]
point = level + trend * horizon
# Conformal-style half-width from recent residual spread, scaled to coverage.
resid = np.diff(history)
z = 1.645 if coverage == 0.90 else 1.96 # 90% -> 1.645, 95% -> 1.96
delta = z * resid.std(ddof=1)
return {"point": float(point), "lo": float(point - delta),
"hi": float(point + delta), "delta": float(delta)}
def calc_tool(expression):
"""Calculator tool: exact arithmetic the language model should not improvise."""
return {"value": float(eval(expression, {"__builtins__": {}}, {}))}
TOOLS = {"forecast": forecast_tool, "calc": calc_tool}
# --- The agent: a plan-and-execute loop with an uncertainty gate. ---
def trust_gate(fc, tau=0.15):
"""Subsection-3 gate: trust the forecast only if the interval is tight enough."""
return abs(fc["delta"]) / max(abs(fc["point"]), 1e-9) <= tau
def plan_and_execute(history, threshold=1000.0, tau=0.15, max_steps=5):
trace = [] # the agent's reasoning + action log
# PLAN: a fixed three-step plan for this goal (a planner LLM would emit this).
plan = ["forecast the demand",
"gate on the calibrated interval",
"if trusted and above threshold, compute the pre-order quantity"]
trace.append(("thought", "plan: " + " | ".join(plan)))
# EXECUTE step 1: call the forecasting tool.
fc = TOOLS["forecast"](history, horizon=1, coverage=0.90)
trace.append(("action", "forecast(history)"))
if "error" in fc: # treat tool error as an observation
trace.append(("observation", "tool error: " + fc["error"]))
return {"decision": "abort", "reason": fc["error"], "trace": trace}
trace.append(("observation", "point=%.0f interval=[%.0f, %.0f]"
% (fc["point"], fc["lo"], fc["hi"])))
# EXECUTE step 2: the uncertainty gate (verification before acting).
if not trust_gate(fc, tau):
trace.append(("observation", "interval too wide (rel %.2f > %.2f): DECLINE"
% (fc["delta"] / abs(fc["point"]), tau)))
return {"decision": "decline", "reason": "forecast not trustworthy",
"trace": trace}
if fc["lo"] <= threshold: # interval straddles threshold -> not actionable
trace.append(("observation", "lower bound %.0f below threshold: DECLINE"
% fc["lo"]))
return {"decision": "decline", "reason": "below threshold at coverage",
"trace": trace}
trace.append(("observation", "passes gate (rel %.2f <= %.2f); lower bound %.0f above threshold"
% (fc["delta"] / abs(fc["point"]), tau, fc["lo"])))
# EXECUTE step 3: compute the pre-order quantity with the calculator tool.
qty = TOOLS["calc"]("%f - %f" % (fc["point"], threshold))["value"]
trace.append(("action", "calc(point - threshold)"))
trace.append(("observation", "pre-order quantity = %.0f units" % qty))
return {"decision": "pre-order", "quantity": qty, "trace": trace}
rng = np.random.default_rng(1)
hist = list(800 + 40 * np.arange(12) + rng.normal(0, 25, size=12)) # rising demand
result = plan_and_execute(hist)
for kind, msg in result["trace"]:
print("%-12s %s" % (kind.upper(), msg))
print("DECISION:", result["decision"], result.get("quantity", ""))
forecast_tool, calc_tool) carry typed contracts; the planner emits a three-step plan; the executor runs each step, handles a tool error as an observation, applies the subsection-3 uncertainty gate trust_gate as a verification step, and only then calls the calculator. Every mechanism of the section (planning, tool calls, the gate, error handling) is visible in one loop.THOUGHT plan: forecast the demand | gate on the calibrated interval | if trusted and above threshold, compute the pre-order quantity
ACTION forecast(history)
OBSERVATION point=1289 interval=[1187, 1391]
OBSERVATION passes gate (rel 0.08 <= 0.15); lower bound 1187 above threshold
ACTION calc(point - threshold)
OBSERVATION pre-order quantity = 289 units
DECISION: pre-order 289.0
The numeric payoff of tool use is sharp here. Without the forecasting tool, the language model would have to guess next month's demand from the history printed in its prompt, and a guess of, say, "about 1200" carries no interval at all, so the agent could neither gate nor justify it. With the tool, the agent gets $1289 \pm 102$ at calibrated 90 percent coverage, a number it can check, gate, and act on. Code 31.3.2 makes the gate's teeth visible by feeding the same agent a volatile history whose interval is too wide to trust.
# Same agent, a noisier and flatter history: the interval should widen and veto.
hist_volatile = list(950 + rng.normal(0, 220, size=12)) # high noise, no clear trend
result2 = plan_and_execute(hist_volatile, threshold=1000.0, tau=0.15)
for kind, msg in result2["trace"]:
print("%-12s %s" % (kind.upper(), msg))
print("DECISION:", result2["decision"], "->", result2["reason"])
THOUGHT plan: forecast the demand | gate on the calibrated interval | if trusted and above threshold, compute the pre-order quantity
ACTION forecast(history)
OBSERVATION point=982 interval=[472, 1492]
OBSERVATION interval too wide (rel 0.52 > 0.15): DECLINE
DECISION: decline -> forecast not trustworthy
Step back and read what the two runs establish together. Code 31.3.1 planned a multi-step goal, called a forecasting tool and a calculator, gated on the calibrated interval, and produced an actionable pre-order quantity. Code 31.3.2 changed only the data, watched the interval widen, and watched the same gate turn a confident-looking forecast into a principled refusal. The pedagogical payoff is that a temporal agent is not a black box that emits answers: it plans visibly, acts through tools, and refuses to act when its own uncertainty quantification says it should not. Now the library equivalent, which collapses the hand-built executor loop and schema plumbing to a few lines.
The from-scratch agent of Code 31.3.1 ran about 45 lines: the executor loop, the trace bookkeeping, the error handling, and the manual dispatch over the tool table. A modern agent framework collapses the planning loop, the structured-schema generation, the tool dispatch, and the trace logging to a few lines, leaving you to write only the tools themselves (which you keep, including the all-important uncertainty gate). With LangChain the same agent becomes roughly:
from langchain.agents import tool, initialize_agent, AgentType
from langchain_openai import ChatOpenAI
@tool
def forecast(history: str) -> str:
"""Forecast next-month demand with a calibrated 90% interval. Input: comma-separated history."""
fc = forecast_tool([float(x) for x in history.split(",")]) # reuse our tool + gate
if "error" in fc: return fc["error"]
ok = trust_gate(fc)
return f"point={fc['point']:.0f} interval=[{fc['lo']:.0f},{fc['hi']:.0f}] trustworthy={ok}"
agent = initialize_agent([forecast], ChatOpenAI(model="gpt-4o-mini"),
agent=AgentType.OPENAI_FUNCTIONS) # ReAct + schema handled internally
agent.run("Forecast demand from 800,840,..., and tell me whether to pre-order above 1000 units.")
The line count drops from roughly 45 (executor loop plus dispatch plus trace) to about 8, with the ReAct loop, the JSON tool schema generated from the function signature and docstring, the structured-call parsing, the tool dispatch, and the reasoning trace all handled internally by the framework. What you must still write yourself, because no framework knows your decision problem, is the tool body and the calibrated-uncertainty gate: the framework automates the plumbing, not the judgment.
Who: A reliability-engineering team at a wind-farm operator, building an agent over the industrial-telemetry series threaded through Chapter 34, with on-call engineers as the humans in the loop.
Situation: Vibration and temperature sensors on each turbine stream continuously; when an anomaly fires (the change-point detector of Chapter 8), an engineer must decide within minutes whether to schedule maintenance, and a wrong call is expensive either way (a needless truck roll or an unplanned failure).
Problem: The team wanted an agent that, on an anomaly, would forecast remaining useful life, estimate the causal effect of deferring maintenance one week, and recommend an action, but only when it could justify the recommendation quantitatively.
Dilemma: A purely generative agent would confidently recommend an action from the anomaly description alone, with no forecast and no uncertainty, exactly the over-confident guessing the chapter warns against. A purely classical planner had no domain file for "diagnose a novel sensor anomaly". Neither alone was acceptable.
Decision: They built a ReAct agent with three tools (a remaining-useful-life forecaster from Chapter 14, a conformal interval from Chapter 19, and a causal deferral-effect estimator from Chapter 30) and gated every quantitative step on its calibrated interval, escalating to a human whenever a gate tripped.
How: The agent planned (forecast life, estimate deferral effect, recommend), executed each step as a validated tool call, and applied the subsection-3 gate $\delta/|\hat{y}| \le \tau$; when the remaining-life interval was too wide to act on, the agent declined to auto-recommend and paged the on-call engineer with the interval attached, exactly the decline branch of Code 31.3.2.
Result: Auto-recommendations were issued only on tight, trustworthy forecasts and were right far more often than the previous heuristic; the wide-interval cases that the old system had quietly guessed on were now routed to humans with the uncertainty made explicit, which the engineers reported as the feature they trusted most.
Lesson: The value of a temporal agent is not that it always answers; it is that it answers when its own calibrated uncertainty says it may, and escalates when it may not. Tool use supplies the quantitative answer; the uncertainty gate supplies the discipline to know when to use it.
6. Hierarchical Temporal Agent Memory: Beyond the Context Window Advanced
The planning and tool-use machinery of this section gives the agent the ability to act. The memory architecture of Section 31.2 gives it short-term context. A third challenge appears only when the agent operates over long time horizons: an agent monitoring a sensor fleet for weeks, or tracking a financial position across hundreds of trading sessions, accumulates far more observations than any context window can hold. The naive solution is to drop old observations. The cost is severe: the agent loses access to past anomalies, long-horizon trends, and prior decisions that may be directly relevant to the current situation. This subsection develops the architectural answer, hierarchical temporal memory, starting from the MemGPT framework (NeurIPS 2023) and extending to adaptive importance-weighted memory management (A-MEM, 2025).
The core problem is precise. Suppose the agent receives one observation per hour. After $K = 100$ observations the context window fills. At hour 101 a new observation arrives. The agent must choose which 99 of its 100 in-context observations to retain and which to evict. If it evicts uniformly (oldest first), it forgets an anomaly from six days ago that is directly analogous to today's reading. If it evicts randomly, it incoherently fragments its temporal record. A principled memory architecture must answer three design questions: where observations go when they leave the context, how to retrieve the right past observations at query time, and which observations to evict or compress when storage itself is bounded.
MemGPT (2023) observes that operating systems solved exactly this problem for programs: main memory (RAM) holds the working set, disk holds the rest, and the OS moves pages between tiers transparently. MemGPT applies the same two-tier architecture to LLM agents. The context window is main memory (fast, limited, always visible to the model). An external vector database is the disk (large, indexed, accessed by explicit retrieval calls). The agent manages its own memory by issuing two function calls: RECALL(query) fetches semantically relevant past observations from the external store into the context, and ARCHIVE(content) moves content from the context to the external store. Unlike an OS, the agent issues these calls explicitly as part of its reasoning, so the memory management is inspectable and auditable.
MemGPT Architecture: Two-Tier Temporal Memory
The MemGPT memory system has two tiers with distinct access semantics. The main context tier contains the agent's current system prompt, the recent observation buffer (the last $K$ observations the agent has seen), the current task state, and any observations retrieved in the current turn. Everything in the main context is visible to the LLM at inference time. The external storage tier contains all observations and facts that have been archived. External storage is a vector database augmented with a timestamp index: each stored item carries both a dense embedding (for semantic search) and a Unix timestamp (for temporal filtering). This dual index enables hybrid retrieval: a query like "anomalies similar to today's reading, from the last 30 days" is executed as a vector similarity search filtered to $t \ge t_{\text{now}} - 30 \text{ days}$.
The agent manages movement between tiers via two function calls that appear in its tool schema alongside domain tools like the forecaster of Section 31.3.3. ARCHIVE$(c)$ serializes content $c$ to a JSON record with fields $\{$text, embedding, timestamp$\}$ and writes it to the vector DB, then removes $c$ from the context. RECALL$(q, k, t_{\text{start}}, t_{\text{end}})$ encodes query $q$ as a dense vector, retrieves the top-$k$ records by cosine similarity from the time window $[t_{\text{start}}, t_{\text{end}}]$, and injects the retrieved texts into the current context. The LLM decides when to call each function: it is trained (or prompted) to ARCHIVE old observations before the context overflows and to RECALL before answering any question that requires long-horizon context.
A-MEM: Adaptive Importance-Weighted Memory
A-MEM (2025) extends MemGPT with an importance scoring function that governs both retrieval ranking and eviction. Each memory record at time $t$ has an importance score
$$I(m, t) = w_1 \cdot \text{recency}(m, t) + w_2 \cdot \text{relevance}(m, t, q) + w_3 \cdot \text{surprise}(m),$$where recency$(m, t) = \exp(-\lambda(t - t_m))$ decays exponentially with age, relevance$(m, t, q)$ is the cosine similarity between the stored embedding and the current query embedding, and surprise$(m)$ is a novelty score computed at archive time (for example, the reconstruction error of a local autoencoder trained on the recent stream). The weights $w_1, w_2, w_3$ are hyperparameters that encode the relative value of fresh versus semantically close versus unusual observations. During retrieval, records are ranked by $I(m, t)$ rather than by raw cosine similarity alone, so a moderately similar but very recent observation outranks a highly similar but ancient one. During eviction, records with $I(m, t) < \theta$ are candidates for compression (their text is replaced by a LLM-generated summary) or deletion. Compression rather than deletion is preferable for temporal AI agents because aggregate patterns over long windows (daily maximums, weekly anomaly counts) are often more useful for trend detection than individual raw observations.
An agent monitors a single sensor at hourly resolution for 30 days: $30 \times 24 = 720$ observations total. The context window holds $K = 100$ observations. After day 5 (observation 120), the context is full and ARCHIVE must be called for at least 20 observations before the next one can be ingested. After 30 days, the external store holds $720 - 100 = 620$ archived observations; the context holds the most recent 100.
The query "any anomalies like today's spike in the past month?" is executed as: (1) encode the current spike observation as a dense vector $\mathbf{q}$; (2) call RECALL($\mathbf{q}$, $k=5$, $t_{\text{start}} = t_{\text{now}} - 30\text{d}$, $t_{\text{end}} = t_{\text{now}}$); (3) the vector DB returns the top-5 archived records by cosine similarity within the 30-day window. Suppose the two most similar records are from day 8 at 14:00 and day 21 at 09:00, with similarities 0.91 and 0.87. These two are injected into the context alongside today's observation. The agent can now reason: "similar spikes occurred on day 8 and day 21; both were followed by a return to baseline within 4 hours; this spike is likely transient." Without MemGPT-style memory, observations from day 8 and day 21 would have been permanently lost to context overflow 3 and 19 days earlier, respectively.
Temporal AI Design Choices
Three design decisions determine the quality of a MemGPT-style temporal memory system. The first is granularity: should raw observations be archived, or should the agent first compress them into summaries? Raw observations preserve all detail but fill storage quickly; summaries (e.g., hourly maximums compressed to daily records) scale to longer horizons at the cost of detail. A practical pattern is a two-stage pipeline: raw observations are archived for the first 7 days, then a background process compresses them to daily summaries for retention up to 90 days. The second decision is the eviction policy: least-recently-used (LRU) eviction is simple but discards old anomalies that are infrequent by definition; importance-weighted eviction (A-MEM) retains surprises but requires calibrating the surprise scorer. The third decision is the retrieval strategy: semantic-only retrieval finds similar observations regardless of age, which is appropriate for pattern matching but wrong for trend detection; semantic-plus-time retrieval (the RECALL API above) supports both uses by allowing the caller to specify the time window explicitly.
"""MemGPT-style temporal memory manager sketch.
Requires: pip install letta chromadb numpy
letta is the production MemGPT package; chromadb is the vector DB backend.
"""
import time
import numpy as np
from dataclasses import dataclass, field
from typing import Any
@dataclass
class MemoryRecord:
text: str
embedding: np.ndarray
timestamp: float # Unix epoch seconds
importance: float = 1.0
class TemporalMemoryManager:
"""Two-tier MemGPT-style memory manager with timestamp-filtered retrieval."""
def __init__(self, context_limit: int = 100, embed_fn=None):
self.context_limit = context_limit
self.context: list[MemoryRecord] = [] # main context (in-window)
self.external: list[MemoryRecord] = [] # external store (archived)
# embed_fn: str -> np.ndarray; replace with a real encoder in production
self.embed_fn = embed_fn or (lambda text: np.random.randn(64))
# ------------------------------------------------------------------
# Core memory management primitives
# ------------------------------------------------------------------
def ingest(self, text: str) -> None:
"""Add a new observation; auto-archive oldest if context is full."""
if len(self.context) >= self.context_limit:
self._archive_oldest()
record = MemoryRecord(
text=text,
embedding=self.embed_fn(text),
timestamp=time.time(),
)
self.context.append(record)
def archive(self, record: MemoryRecord) -> None:
"""Move a record from the context to external storage."""
if record in self.context:
self.context.remove(record)
self.external.append(record)
def recall(self, query: str, k: int = 5,
t_start: float = 0.0, t_end: float = float("inf")
) -> list[MemoryRecord]:
"""Retrieve top-k external records by cosine similarity within [t_start, t_end]."""
q_emb = self.embed_fn(query)
candidates = [r for r in self.external
if t_start <= r.timestamp <= t_end]
if not candidates:
return []
# Cosine similarities
embs = np.stack([r.embedding for r in candidates])
sims = embs @ q_emb / (
np.linalg.norm(embs, axis=1) * np.linalg.norm(q_emb) + 1e-9
)
top_k_idx = np.argsort(sims)[::-1][:k]
retrieved = [candidates[i] for i in top_k_idx]
# Inject retrieved records into context temporarily
self.context.extend(retrieved)
return retrieved
# ------------------------------------------------------------------
# A-MEM importance scoring (simplified)
# ------------------------------------------------------------------
def importance(self, record: MemoryRecord, query_emb: np.ndarray,
lambda_decay: float = 0.01,
w: tuple[float, float, float] = (0.4, 0.4, 0.2)) -> float:
t_now = time.time()
recency = np.exp(-lambda_decay * (t_now - record.timestamp))
relevance = float(
record.embedding @ query_emb
/ (np.linalg.norm(record.embedding) * np.linalg.norm(query_emb) + 1e-9)
)
surprise = record.importance # pre-computed at archive time
return w[0] * recency + w[1] * relevance + w[2] * surprise
# ------------------------------------------------------------------
# Internal helpers
# ------------------------------------------------------------------
def _archive_oldest(self) -> None:
"""Evict the oldest context record to external storage."""
oldest = min(self.context, key=lambda r: r.timestamp)
self.archive(oldest)
# --- letta (production MemGPT) shortcut ---
# from letta import create_client
# client = create_client()
# agent = client.create_agent(name="sensor-monitor")
# # Ingest an observation
# client.send_message(agent_id=agent.id, message="sensor reading: 42.7 at 14:00", role="user")
# # The agent automatically manages RECALL / ARCHIVE via its built-in memory tools
ingest adds observations and auto-archives when the context limit is reached; recall performs time-filtered cosine-similarity retrieval from external storage; importance implements the A-MEM scoring function. The commented block shows the equivalent letta production API.A 2025 survey of temporal agent memory systems finds that MemGPT-style two-tier architectures have become the de facto standard for long-horizon LLM agents but identifies three open problems. First, the context-to-external handoff point is almost always a fixed count threshold ($K$ observations), yet the informationally correct handoff depends on the entropy of the incoming stream: a burst of anomalies should be retained longer than a quiet baseline period. Second, retrieval latency in vector DBs grows sub-linearly but non-trivially with archive size, and at the scale of months of hourly data the RECALL latency can exceed the agent's allowed response time. Third, the importance scoring of A-MEM requires a calibrated surprise detector, and calibration drift over long deployment periods causes the system to mis-classify routine events as novel. The survey recommends drift-aware recalibration of the importance scorer every 7 days using the preceding window's distribution, an explicit pattern that practitioners should build into any production temporal agent memory system.
For each of the following tasks, decide whether classical search/PDDL planning or LLM plan-and-execute is the better fit, and justify your choice using the dimensions of Table 31.3.1 (size and knownness of the action space, availability of a transition model, need for guarantees): (a) scheduling jobs on five machines to minimize makespan; (b) deciding whether to hedge a currency position given a forecast and a news feed; (c) stacking colored blocks into a named configuration; (d) triaging an unfamiliar production incident from logs. In one sentence per task, name the single dimension that decides the answer.
Extend Code 31.3.1 with a third tool causal_tool(history, intervention) that returns an estimated treatment effect with a confidence interval (you may stub the estimate, but it must return a point and an interval). Add a fourth plan step that, after the demand forecast passes its gate, calls the causal tool to estimate the effect of a five percent price increase, gates that effect on its own interval with the same trust_gate, and only adjusts the pre-order quantity if both gates pass. Print a trace showing both gates and confirm the agent declines if either interval is too wide.
The tolerance $\tau$ in the gate $\delta/|\hat{y}| \le \tau$ trades two error costs: acting on a bad forecast (over-confidence) versus declining a good one (over-caution). Build a small simulation that draws many demand histories of varying volatility, runs plan_and_execute at several values of $\tau$, and scores each $\tau$ against a cost model where a wrong pre-order costs $C_{\text{act}}$ and a needless decline costs $C_{\text{decline}}$. Plot total cost against $\tau$, identify the cost-minimizing tolerance, and explain how it shifts as you raise $C_{\text{act}}$ relative to $C_{\text{decline}}$. Connect your finding to the coverage-level choice of Chapter 19.
We have given the agent the two halves of acting in an open world: a plan that decomposes a goal into steps, and tools that let each step reach outside the frozen weights to a calculator, a database, and (the temporal payoff) the forecasters and causal estimators of the earlier chapters, every quantitative call gated on the calibrated uncertainty of Chapter 19. What we have not yet addressed is the horizon. The agent of this section planned three steps and acted; real temporal tasks unfold over long horizons where a reward or a consequence arrives many steps after the action that caused it, and the agent must assign credit across that delay to learn which of its earlier choices mattered. Section 31.4 takes up exactly this: long-horizon tasks and the credit-assignment problem, where the sequential-decision machinery of Part VI meets the open-ended action space of the agent, and the question becomes not just "what should I do now" but "which of the many things I did is responsible for how this turned out".