"They showed me the curve and it was beautiful: loss sliding down a straight line on a log-log plot, forever, as long as someone kept feeding me parameters and tokens and GPU-hours. Nobody mentioned that the line is straight precisely because it never quite reaches the floor, and that the bill is not."
A Loss Curve That Keeps Going Down if You Just Keep Paying
A scaling law is an empirical promise written as a power law: hold everything else fixed, grow one resource (parameters $N$, training tokens $D$, or compute $C$), and the validation loss falls along a straight line on a log-log plot, $L(N) \approx L_\infty + a\,N^{-\alpha}$. For large language models this promise has held across more than five orders of magnitude, and it turned model building into a budgeting exercise: pick a compute budget, read the optimal model and data sizes off the law, and the loss is predictable before a single GPU spins up. This section asks the question that Chapter 36 cannot avoid: does temporal AI obey the same law? The 2024-2026 evidence says partly: time-series foundation models do show clean power-law improvement with model and data scale, but they hit a data wall far sooner than text does, because the world simply contains less high-quality, diverse temporal data than it contains text, and what exists is more heterogeneous. So the section develops three things in tension. First, the scaling laws themselves and the power-law mathematics behind them. Second, the data bottleneck specific to temporal AI and the synthetic-data response. Third, the efficiency frontier: the accuracy-versus-cost Pareto front on which tiny models, linear-time state-space models, and inference-time search all compete, and where the book's recurring lesson, that bigger is not automatically better for time series, finally gets quantified. You leave able to fit a scaling curve from scratch, extrapolate it, and price the next order of magnitude of accuracy.
This chapter has been about what a general temporal intelligence might look like. Section 36.3 argued that foundation models, agents, and sequence-models-as-policies converge into one generalist agent that perceives, predicts, and acts. This section confronts the economics of getting there. The defining discovery of the modern era of deep learning is not any single architecture but a regularity about all of them: performance improves with scale in a way so smooth and so predictable that it can be written as a power law and extrapolated. We met the consequences of that regularity already, in the temporal foundation models of Chapter 15, which exist only because someone bet that scaling a sequence model on enough time series would pay off the way it paid off for text. Here we examine the bet itself: when it holds for temporal data, when it breaks, and what to do at the frontier where adding parameters stops being the cheapest way to buy accuracy. We use the unified notation of Appendix A: $N$ for parameter count, $D$ for dataset size in tokens or observations, $C$ for training compute, $L$ for the loss.
The four competencies this section installs are these: to read and write a power-law scaling relation and understand what its exponent and irreducible-loss floor mean; to explain why temporal pretraining corpora are smaller and more heterogeneous than text and what synthetic data does about it; to locate a model on the accuracy-versus-cost Pareto front and argue from that front why a tiny linear model can dominate a large transformer for a given task; and to fit a scaling curve to real measurements, from scratch and with a library, and use it to predict both the error and the cost at a larger scale. These are the skills that turn "we should make it bigger" from a reflex into a calculation.
1. Scaling Laws: Model, Data, Compute, and the Power-Law Loss Curve Intermediate
The empirical scaling law is the observation that, over a wide range, the validation loss of a well-trained neural sequence model is a power law in each of the three resources you can spend on it. Holding the other two abundant, the loss as a function of parameter count $N$, dataset size $D$, and compute $C$ follows
$$L(N) \approx L_\infty + a\,N^{-\alpha}, \qquad L(D) \approx L_\infty + b\,D^{-\beta}, \qquad L(C) \approx L_\infty + c\,C^{-\gamma}.$$Here $L_\infty$ is the irreducible loss, the floor set by the intrinsic entropy of the data that no amount of scaling can cross (for a forecasting task, the genuine unpredictability we bounded back in Chapter 1); the coefficients $a, b, c$ set the height of the curve; and the exponents $\alpha, \beta, \gamma$, typically small positive numbers in the range $0.05$ to $0.5$, set how fast the loss falls. The signature of a power law is that, after subtracting the floor, $\log(L - L_\infty)$ is linear in $\log N$ with slope $-\alpha$. That straight line on a log-log plot is the whole phenomenon, and it is why a few small training runs can predict the loss of a run a hundred times larger: you fit the line on the cheap points and read off the expensive one.
Why a power law and not, say, an exponential? The deep reason is still debated, but the practical content is empirical and robust: across vision, language, and increasingly time series, the loss-versus-scale relation is well described by a power law with a floor over the range that matters for budgeting. Two consequences follow immediately and are worth stating as the working grammar of the field. First, because the exponent is small, returns diminish hard: with $\alpha = 0.1$, cutting the reducible loss in half requires multiplying $N$ by $2^{1/0.1} = 2^{10} \approx 1000$. Accuracy is bought in orders of magnitude, not increments. Second, because the curve is smooth and the floor is real, there is a compute-optimal allocation: for a fixed compute budget $C \approx 6 N D$ (the rule of thumb for transformer training cost), there is a best split between making the model bigger and training it on more data, and the Chinchilla result (Hoffmann et al., 2022) found that for text the two should grow in roughly equal proportion. The optimal $N$ and $D$ both scale as a power of the budget, $N_\text{opt} \propto C^{p}$, $D_\text{opt} \propto C^{q}$, and this is the calculation that decides how to spend a training run.
A scaling law has two numbers that matter and they answer different questions. The exponent $\alpha$ answers "how expensive is the next increment of accuracy?": a small exponent means accuracy is costly, bought only in multiplicative jumps of scale, and it never improves. The floor $L_\infty$ answers "how good can this ever get?": no matter how much you spend, the loss asymptotes to the irreducible entropy of the task, the unpredictability that Chapter 1 argued is intrinsic to many temporal processes. Confusing the two is the classic scaling mistake. A team that sees loss still falling and concludes "just scale more" may be paying a steep exponent to crawl the last few percent toward a floor that a different model, or a different data source, would have lowered outright. Read both numbers before you sign the GPU invoice.
Now the temporal question. Do time-series foundation models scale like LLMs? The 2024-2026 evidence is that they scale, but with two important asterisks. On the positive side, the foundation forecasters of Chapter 15 show clean power-law improvement: TimesFM (Das et al., 2024), Moirai (Woo et al., 2024), Chronos (Ansari et al., 2024), and MOMENT (Goswami et al., 2024) all report monotone gains from more parameters and more pretraining series, and zero-shot accuracy that improves with scale across heterogeneous benchmarks, the same qualitative curve LLMs trace. On the asterisk side, the exponents are often shallower and the curves saturate sooner than for text, and the dominant reason is data. The total quantity of diverse, high-quality temporal data on Earth is far smaller than the total quantity of text, and unlike text it is wildly heterogeneous in sampling rate, dimensionality, scale, and domain, so a token of one series teaches less about another than a token of English teaches about more English. The result is that temporal scaling laws tend to hit the data-limited regime, where loss is bounded by $L(D)$ rather than $L(N)$, while the model could still absorb more parameters if only there were more data to feed them. That data wall, not the model, is the binding constraint, and it is the subject of subsection two.
2. The Temporal Data Bottleneck and Synthetic-Data Scaling Intermediate
The single most important way temporal AI differs from language modeling at the frontier is the supply of data, so it deserves its own treatment. Language models are trained on a corpus measured in trillions of tokens scraped from a civilization that has been writing text at scale for decades, and the bottleneck there is only now coming into view. Temporal AI has no comparable reservoir. The world's time series are siloed inside firms and instruments, frequently proprietary, irregularly sampled, of wildly varying dimensionality and scale, and they do not transfer across domains the way natural language transfers across topics: an electrocardiogram, an electricity-demand trace, and a foreign-exchange tape share almost no surface statistics. So the effective pretraining corpus for a temporal foundation model is both smaller in raw size and more heterogeneous in kind than a text corpus, which is exactly the combination that makes the data exponent $\beta$ bite early: you run out of informative data before you run out of useful parameters.
Two responses dominate the 2024-2026 literature. The first is aggressive aggregation and harmonization: projects like the Moirai pretraining corpus (LOTSA) and the Chronos training mixture assemble hundreds of public datasets into a single tokenized stream, normalizing scale and resampling rate so that heterogeneous series become comparable training examples. This buys real scale, but it is bounded by what public data exists. The second, and the more open-ended, is synthetic data: generate training series from stochastic processes, simulators, and learned generative models, so that the data axis $D$ can be pushed without waiting for the world to produce more measurements. This is the direct callback to Section 17.5, where we built generative temporal models (diffusion and flow models for time series) precisely so they could manufacture realistic synthetic sequences; here that capability becomes a scaling lever. Chronos, for instance, augments its real corpus with synthetic series drawn from Gaussian processes and composed seasonal-trend kernels, and reports that the synthetic fraction measurably improves zero-shot accuracy on unseen domains.
In Section 17.5 we trained generative temporal models with one goal in mind: to sample realistic series, for augmentation, simulation, and privacy. The scaling perspective recasts that same machinery as a response to the data wall. If the binding constraint on a temporal foundation model is the supply of diverse training series, then a generator that produces statistically plausible, controllably varied series is a way to move along the data axis $D$ without new measurements. The catch is the obvious one: synthetic data can only encode the structure its generator knows, so it lifts the floor on diversity but cannot conjure genuinely novel real-world dynamics. It pushes the data exponent further before it saturates; it does not abolish the saturation.
There is a sharp theoretical caveat on the synthetic-data lever, and it is the reason this is a research frontier rather than a solved problem. Training a model on data generated by another model risks model collapse: if the synthetic distribution is narrower than reality, repeated training on it can shrink the diversity the model has access to, so that more synthetic tokens stop helping and eventually hurt. The 2024-2026 consensus is that synthetic data is most valuable as a supplement to real data, mixed in at a controlled fraction and grounded in mechanistic generators (known stochastic processes, physics simulators) rather than purely learned ones, so that it adds coverage without collapsing diversity. The practical upshot for temporal AI is that the data axis is not free but it is also not fixed: a careful synthetic-plus-real mixture can extend the useful range of $D$ by a meaningful factor, and getting that mixture right is one of the live problems of the field.
For LLMs the historical bottleneck was compute and parameters; data felt nearly infinite. For temporal foundation models the situation is inverted: parameters and compute are available, but diverse high-quality time series are scarce, so the binding term in the scaling law is usually $L(D)$, not $L(N)$. This single fact reorganizes priorities. It means the highest-leverage research for temporal scaling is on the data side (harmonizing public corpora, generating trustworthy synthetic series, building better tokenizations that transfer across domains) rather than on the model side. It also means a practitioner staring at a saturated loss curve should suspect a data ceiling before reaching for a bigger model, because past the data-limited point the extra parameters buy almost nothing.
3. The Efficiency Frontier: Accuracy Versus Cost and Why Bigger Is Not Always Better Intermediate
Scaling laws describe how accuracy improves with resources, but accuracy is never the only objective. Every deployed temporal model is also judged on cost: inference latency, memory, energy, and dollars per prediction. Plot accuracy against cost for a family of models and you get a scatter of points; the lower-right boundary of that scatter, the set of models that are not dominated (no other model is both cheaper and more accurate), is the Pareto front. The whole practical art of model selection is choosing a point on that front, and the central lesson of this book, that bigger is not automatically better for time series, is precisely the statement that for many temporal tasks the Pareto-optimal point sits far to the cheap end of the front.
The evidence is concrete and recurring. The Tiny Time Mixers (TTM, Ekambaram et al., 2024) family achieves competitive or superior zero-shot forecasting with under a million parameters, orders of magnitude smaller than the foundation transformers it competes with, and the embarrassingly simple DLinear (Zeng et al., 2023), a single linear layer on the decomposed series introduced in Section 14.5, matches or beats elaborate transformers on a striking fraction of standard long-horizon forecasting benchmarks. The lesson those results teach, and that this book has repeated since Section 15.4, is that many real-world series are dominated by a few simple components (trend, seasonality, a short memory) that a small model captures fully, so the extra capacity of a large model goes into fitting noise rather than signal. On such tasks the large model is not just wasteful, it is often worse, because the irreducible floor $L_\infty$ is reached by the small model and everything beyond is overfitting headroom.
Architecture changes the shape of the cost axis itself, not just where you sit on it. The linear-time state-space models of Chapter 13, Mamba and its kin, occupy a distinguished place on the efficiency frontier because their cost grows linearly with sequence length rather than quadratically like attention, so for long sequences they offer accuracy comparable to a transformer at a fraction of the compute, shifting the entire front toward the cheap end. The efficiency frontier for temporal AI is therefore not a single curve but a layered one: tiny linear models (DLinear, TTM) anchor the cheapest corner for series with simple structure; linear-time SSMs (Chapter 13) dominate the long-sequence regime; and large foundation transformers (Chapter 15) earn their cost only when the task genuinely needs broad cross-domain transfer or complex, high-dimensional dynamics that the small models cannot represent.
Who: A demand-planning team at a grocery chain forecasting daily sales for 40,000 store-product pairs, retrained nightly on a modest CPU cluster.
Situation: A pilot used a fine-tuned foundation transformer from Chapter 15 and reached strong accuracy, but the nightly inference bill and latency were incompatible with refreshing 40,000 series before the morning replenishment cutoff.
Problem: They needed near-foundation accuracy at a cost that fit a nightly CPU batch window, on series that were mostly trend plus weekly and yearly seasonality with promotional spikes.
Dilemma: Stay with the transformer (accurate, far over budget), drop to a classical per-series model (cheap, but no cross-series transfer and weak on sparse items), or try a tiny model on the Pareto front.
Decision: They benchmarked DLinear and TTM against the transformer on a held-out month and found TTM within one percent of the transformer's accuracy at roughly two orders of magnitude less inference cost, with DLinear close behind on the non-promotional items.
How: They deployed TTM as the fleet default and kept the transformer only for the few thousand high-value, high-variance items where its extra accuracy cleared the cost, a Pareto-front split rather than a single global choice.
Result: Nightly inference fit the CPU window with room to spare, accuracy fell within tolerance, and the compute bill dropped by more than ninety percent versus the all-transformer pilot.
Lesson: The right question is never "which model is most accurate?" but "which point on the Pareto front meets my accuracy bar at the least cost?", and for simply-structured series that point is usually small. Reach for the foundation model only where the structure genuinely exceeds what a tiny model can represent.
4. Inference-Time Scaling: Buying Accuracy With Compute at Test Time Advanced
Parameter scaling is not the only axis along which accuracy can be bought. A second, increasingly important lever spends compute at inference time rather than at training time: instead of a larger model that thinks once, use a fixed model that thinks longer, searching, sampling, or planning before it commits to an output. This inference-time (or test-time) scaling has its own loss-versus-compute curve, and in some regimes it is a cheaper way to buy accuracy than growing the model, because the extra compute is spent only on the inputs that need it and only when they are queried, not baked permanently into every forward pass.
For temporal AI the natural home of inference-time scaling is sequential decision making, and we built its machinery in Chapter 29. There, planning with a learned world model (Monte Carlo tree search, model-predictive control, trajectory sampling) is exactly inference-time scaling: a fixed-size dynamics model is queried many times to roll out and compare candidate futures, and accuracy of the chosen action improves with the search budget along its own scaling curve. The same idea appears in probabilistic forecasting as scenario sampling: draw many sample paths from a generative forecaster and aggregate, trading inference compute for a sharper estimate of the predictive distribution. In both cases the model is frozen and the knob is "how much test-time compute do I spend per query?".
Training compute and inference compute are separate budgets with separate scaling curves, and the optimal mix depends on how often you query. A larger model pays its cost once at training and then on every single inference forever; inference-time search pays nothing extra at training but pays per query. So for a model served billions of times, amortized parameter scaling can win; for a model queried rarely but needing high accuracy when it is (a high-stakes planning decision, a risk forecast under stress), inference-time scaling is often the cheaper path to the same accuracy. The frontier question for general temporal intelligence is not "train-time or test-time?" but "given my query volume and accuracy bar, what is the cost-minimizing split between the two budgets?"
The 2024-2026 prominence of inference-time scaling in language models (chains of reasoning, best-of-$n$ sampling, verifier-guided search) has a direct temporal analogue that is only beginning to be explored: spending test-time compute to let a temporal agent plan, simulate, and verify before acting. It reframes the scaling story of this chapter. The road to more capable temporal AI is not only a wider model trained on more data; it is also a fixed model given more room to deliberate at the moment of decision, and the two roads have different cost structures that a deployment should price against each other rather than assume the first.
5. Worked Example: Fitting and Extrapolating a Scaling Law Advanced
We now make the scaling law executable. The plan is to take a set of (model-size, validation-error) measurements from small, cheap training runs, fit the power law $L(N) = L_\infty + a\,N^{-\alpha}$ to them from scratch, and then use the fitted law to extrapolate the error and the cost at a model size we never trained, the exact reasoning a lab uses to decide whether the next order of magnitude is worth it. We fit in log space because a power law (after subtracting the floor) is linear there, so an ordinary least-squares line recovers the exponent. Code 36.4.1 does the from-scratch fit.
import numpy as np
# Measured (model size N, validation error L) from six small, cheap training runs.
# Synthetic but realistic: error falls as a power law toward an irreducible floor.
N = np.array([1e5, 3e5, 1e6, 3e6, 1e7, 3e7]) # parameter counts
L = np.array([0.182, 0.151, 0.128, 0.112, 0.101, 0.094]) # validation error (e.g. normalized MAE)
L_inf = 0.078 # irreducible-loss floor: the task's intrinsic unpredictability (Chapter 1)
y = np.log(L - L_inf) # subtract floor, then a power law becomes a straight line in log-log
x = np.log(N)
# Ordinary least squares for the line y = m*x + b: slope m = -alpha, intercept b = log(a).
m, b = np.polyfit(x, y, 1)
alpha = -m
a = np.exp(b)
print("fitted exponent alpha = %.4f" % alpha)
print("fitted coefficient a = %.4f" % a)
# Extrapolate: predicted error at a model 10x larger than the biggest we trained.
N_target = 3e8 # 10x the largest measured size
L_pred = L_inf + a * N_target ** (-alpha) # the scaling-law prediction
print("predicted error at N=3e8: L = %.4f" % L_pred)
np.polyfit of degree 1) recovers the exponent $\alpha$ as the negative slope and the coefficient $a$ as the exponentiated intercept. The fitted law then extrapolates the error at a size ten times larger than any training run.fitted exponent alpha = 0.1873
fitted coefficient a = 0.6042
predicted error at N=3e8: L = 0.0890
The from-scratch version exposes the mechanics (subtract the floor, fit a line in log space) but it assumes we know $L_\infty$ and it cannot fit the floor jointly. A library solves the full nonlinear three-parameter fit directly. Code 36.4.2 uses scipy.optimize.curve_fit to fit $L_\infty$, $a$, and $\alpha$ together in one call.
from scipy.optimize import curve_fit
def power_law(N, L_inf, a, alpha):
return L_inf + a * N ** (-alpha) # the full three-parameter scaling law
# Fit all three parameters jointly; p0 is a rough starting guess.
params, _ = curve_fit(power_law, N, L, p0=(0.07, 0.6, 0.19), maxfev=10000)
L_inf_fit, a_fit, alpha_fit = params
print("joint fit: L_inf=%.4f a=%.4f alpha=%.4f" % (L_inf_fit, a_fit, alpha_fit))
print("predicted error at N=3e8: L = %.4f" % power_law(3e8, *params))
scipy.optimize.curve_fit fits all three parameters (floor $L_\infty$, coefficient $a$, exponent $\alpha$) jointly via nonlinear least squares, removing the manual floor-subtraction and log-transform of Code 36.4.1. The two-line call replaces roughly a dozen lines of manual setup and, unlike the from-scratch version, estimates the irreducible floor from the data instead of assuming it.joint fit: L_inf=0.0796 a=0.5821 alpha=0.1842
predicted error at N=3e8: L = 0.0892
The from-scratch fit of Code 36.4.1 ran about a dozen lines and required us to know the irreducible floor $L_\infty$ in advance so we could subtract it before the log-transform. scipy.optimize.curve_fit collapses that to a two-line call that fits the floor, coefficient, and exponent jointly by nonlinear least squares, handling the optimization, the Jacobian, and the parameter covariance internally. The line-count reduction is real but the substantive win is that the library estimates the floor from the data, which is exactly the parameter you most want and least know. For confidence intervals on the extrapolation, the same call returns the parameter covariance matrix as its second output, which a from-scratch version would need many more lines to produce.
Use the fitted law to price a decision. The largest trained model has $N = 3 \times 10^7$ parameters and error $0.094$; the fit predicts a tenfold larger model, $N = 3 \times 10^8$, reaches error $0.089$, an absolute improvement of $0.005$ (about five percent of the reducible gap above the $0.078$ floor). Now the cost. Training compute scales as $C \approx 6 N D$, and at the compute-optimal allocation data grows with the model, so a tenfold larger model trained compute-optimally costs on the order of $10 \times 10 = 100$ times the compute of the current run (model up tenfold, data up tenfold). If the current run cost \$2{,}000 in GPU time, the tenfold model costs on the order of \$200{,}000 to buy a $0.005$ error reduction. Whether that is worth it is a business question, not a machine-learning one, but the scaling law is what turns the question into a number: it converts "should we scale?" into "are five millis of error worth two hundred thousand dollars?", which a decision-maker can actually answer. That conversion, from vague ambition to a priced increment, is the entire practical value of a scaling law.
Read the three pieces together. Code 36.4.1 fit the power law by hand and exposed its mechanics; Code 36.4.2 fit it with a library that recovered the floor from data; and the numeric example turned the fitted curve into a dollar figure for the next order of magnitude. That pipeline, fit on cheap runs, extrapolate, then price the increment, is how frontier labs decide where to spend, and it is the discipline this section exists to install: scaling is a calculation, not a reflex.
The empirical scaling of temporal foundation models is an active and unsettled research program. The 2024 foundation forecasters (TimesFM, Moirai, Chronos, MOMENT) each report scaling behavior, but a unified Chinchilla-style compute-optimal law for time series, prescribing how to split a budget between model size and the scarce data, is still being worked out, and the data-limited regime makes the text recipe a poor fit. Several 2024-2025 studies (for example scaling analyses of Chronos and Moirai variants, and the LOTSA and GIFT-Eval benchmark efforts) measure how zero-shot accuracy improves with pretraining corpus size and find shallower, sooner-saturating curves than text, consistent with the data-wall argument of subsection two. On the synthetic-data side, the question of how much synthetic series can substitute for real before model-collapse effects appear is open and consequential. And inference-time scaling for temporal decision making (test-time planning and search budgets, connecting to Chapter 29) is an emerging axis whose own scaling laws are barely charted. The practitioner's 2026 takeaway: temporal AI scales, but its laws are its own, data-bound and shallower than language's, and reading them correctly is worth more than copying the LLM playbook.
Every scaling curve is a quiet morality tale. The exponent makes the early going look heroic, loss tumbling as you scale, and it is genuinely thrilling. Then the floor, the irreducible entropy of a world that simply will not be predicted past a point, asserts itself, and the same curve that promised the moon flattens into a polite asymptote a few percent above the noise. The lesson the whole field keeps relearning is the one Chapter 1 opened with: some of the future is unknowable, and no number of parameters repeals that. Scaling buys you the reducible part faster. It never sells you the irreducible part at any price.
6. Exercises
These exercises move from reading a scaling law, to implementing the fit and the cost calculation, to taking a position on where temporal scaling goes next. The fitting code of subsection five is the starting point for the implementation problem.
Conceptual
- Exponent versus floor. Two temporal foundation models are fit on the same task. Model family A has a steeper exponent $\alpha_A = 0.30$ but a higher floor $L_\infty^A = 0.10$; family B has a shallower exponent $\alpha_B = 0.15$ but a lower floor $L_\infty^B = 0.06$. Sketch both curves on log-log axes and explain which family you would choose if you have a large compute budget and want the lowest achievable error, versus a small budget and want the best error per dollar. State why the crossing point of the two curves is the decision boundary.
- Why temporal data is the bottleneck. Explain in your own words why the scaling law for a temporal foundation model is typically bound by the data term $L(D)$ rather than the parameter term $L(N)$, and contrast this with the historical situation for large language models. Identify two concrete properties of real-world time series (beyond raw quantity) that make a temporal pretraining corpus less informative per token than a text corpus, and connect each to the heterogeneity argument of subsection two.
Implementation
- Fit, extrapolate, and bound. Starting from Code 36.4.2, refit the scaling law and use the parameter covariance matrix (the second return value of
curve_fit) to put an approximate uncertainty band on the extrapolated error at $N = 3 \times 10^8$. Draw 200 parameter samples from a multivariate normal with the fitted mean and covariance, evaluate the predicted error for each, and report the 5th and 95th percentiles. Comment on whether the extrapolation is tight enough to base a \$200,000 scaling decision on, given that the prediction extends a full order of magnitude beyond the largest measured point. - The compute-optimal split. Assume training compute $C \approx 6 N D$ and a simple joint law $L(N, D) = L_\infty + a N^{-\alpha} + b D^{-\beta}$ with $\alpha = 0.18$, $\beta = 0.12$, $a = b = 0.6$, $L_\infty = 0.08$. For a fixed budget $C = 10^{18}$ FLOPs, numerically minimize $L$ over the split between $N$ and $D$ subject to $6ND = C$, and report the optimal $N$ and $D$. Repeat for a temporal setting where data is capped at $D \le D_\text{max}$ (the data wall) and show how the optimal allocation changes once the cap binds.
Open-ended
- Train-time versus test-time scaling for a temporal agent. Consider a temporal control agent (Chapter 29) that can either be made larger (parameter scaling) or given a larger test-time planning budget (inference-time scaling). Design an experiment, on a simulator of your choice, that measures both scaling curves and finds the query volume at which one strategy becomes cheaper than the other for a fixed accuracy target. Discuss what properties of the task (query frequency, stakes per decision, latency budget) push the answer toward one axis or the other, and relate your reasoning to the two-budget insight of subsection four.