Abstract. An agent session re-processes its entire history on every model call. This single architectural fact makes session input-spend quadratic in length, and it holds regardless of whether the session ever compacts: a 700,000-token session on a one-million-token window spends $183 on prefix reads with caching working perfectly, against $21 under a managed working set. We present a census of 2,538 real working sessions establishing where this cost concentrates, a measured reference-distance study showing that 81% of everything a session reaches back for lies within its last 50 turns (8.3k tokens; roughly 33 KB), and a closed-form optimal compaction trigger derived from three measured inputs, landing near 50k tokens: half the common default and an order of magnitude below platform behaviour.
The analytic results are validated against sealed known-answer counterfactuals on three real sessions, reproduced in-band nine of nine, including one session where compacting late lost money (−2.1%). We report the cache-accounting assumptions as explicit bounds rather than estimates, and we describe the per-tenant calibration system that replaces every constant in this paper with a measured value carrying its own provenance.
01Every turn re-reads everything
The transformer inference contract is unforgiving: each call processes the full conversation prefix. A session that has accumulated S tokens pays for those S tokens again on the next turn, and the next. Summed over a session of N turns growing at r tokens per turn, total input processing is approximately r·N²/2. The spend is quadratic in session length.
Prompt caching softens the constant without touching the exponent. Cached prefix reads bill at a tenth of the base input rate, so a perfectly cached session pays the quadratic sum at 0.1×. The exponent survives. So do three complications the vendor pricing pages make explicit: cache writes bill at 1.25×, the default cache lifetime on the raw API is five minutes[6], and two of the three major one-million-token models double their input price for requests beyond 200k tokens[7,8]. A larger window is therefore not an escape from this economics. It is a licence to let the quadratic run longer.
02The census: where sessions actually live
We measured 2,538 working sessions from our own estate, each of at least 20 turns. The median session is small: 33k tokens, 47 turns. It never meets a compaction and its quadratic term is trivial. The tail is another animal. Five percent of sessions exceed 100k tokens; the largest reached 2.4 million tokens across 8,692 turns and compacted 53 times. Among sessions that compact at all, the median compacts twice.
| tokens per session | median | mean | p90 | max |
|---|---|---|---|---|
| all sessions (n=2,538) | 33,095 | 57,116 | 88,238 | 2,399,678 |
The concentration matters commercially. On our own account we observed the largest sessions re-reading roughly half a million tokens per message with caching healthy (97.6% of tokens served as cache reads across 686 live sessions in a six-hour audit). The tail is a small fraction of sessions and the overwhelming fraction of spend. Public data agrees on the shape: coding-lane prompts in the OpenRouter 100-trillion-token study routinely exceed 20k input tokens and have grown roughly four-fold in two years[4], while consumer chat sessions rarely pass five turns[5] and never meet this problem at all.
03The quadratic, with caching granted
The table below computes input-spend for sessions of measured shape (mean 408 tokens added per turn, from the census) at pinned Claude Sonnet 4.6 rates, which offer the full one-million window with no long-context premium. We grant the unmanaged case everything: a 100% cache hit rate, every token written once, no invalidations, no premium. These are bounds, stated as bounds.
| session | turns | uncached (quadratic) | best-case cached | managed (~35k ceiling) | overpay |
|---|---|---|---|---|---|
| 200k | 490 | $147 | $15 | $6 | 2.6–25× |
| 700k | 1,715 | $1,801 | $183 | $21 | 8.9–87× |
| 1M | 2,450 | $3,675 | $371 | $29 | 12.6–125× |
| 2.4M (largest real) | 5,882 | $21,175 | $2,127 | $71 | 30–299× |
The ratio grows with session length, roughly as S/2 divided by the ceiling. This is the finding that inverts the intuitive reading of large context windows: the bigger the window, the more valuable its management. One-million-token contexts do not dodge the quadratic. They let it compound to four figures per session.
04How far back a session actually reaches
A managed ceiling is only safe if the discarded material is genuinely not needed. We measured this directly: 214,175 reference events (a turn re-mentioning a file path seen earlier) across the estate, recording the distance in turns to the previous mention.
| tail depth kept | 10 turns | 25 | 50 | 100 | 200 |
|---|---|---|---|---|---|
| references covered | 54.9% | 70.9% | 80.8% | 88.0% | 92.8% |
| measured window size | ~1.7k tok | ~4.2k | 8.3k (33 KB) | ~17k | ~33k |
The curve has a knee. Fifty turns of tail (about 33 KB of text) covers 81% of everything a session reaches back for; doubling it buys seven points for twice the carrying cost. The remaining 12–19% is deep retrieval, with a p99 of 1,519 turns, unreachable by any affordable tail. That residue defines the division of labour: a compact salient store plus a modest tail carries the working set, and anything deeper is a retrieval problem, not a context problem. For scale: the resulting working context of roughly 25k tokens is about 100 KB, where an unmanaged one-million-token window is four megabytes dragged past the meter on every turn.
05The optimisation curve
Compaction is not free, so triggering it is a trade. Compact early and every turn reads a small prefix, but compaction events (extraction plus re-writing the reinstated floor as fresh cache) come often. Compact late and events are rare while every turn reads a fat prefix. Total spend is U-shaped in the trigger point T, and the minimum has a closed form:
where F is the reinstated floor (salient store plus measured tail; about 25k tokens here), r the measured tokens added per turn, Cevent the measured cost of one compaction (extraction $0.14–$1.62 metered depending on extractor, plus the floor re-write), and cread the effective per-token read rate. With our measured inputs the optimum lands near 50k tokens under cached reads and 33k uncached.
| trigger | 35k | 50k = T* | 100k (our default) | 200k | never (1M) |
|---|---|---|---|---|---|
| session cost | $32.08 | $25.96 | $34.39 | $58.86 | $183 |
| compactions | 70 | 28 | 9.3 | 4 | 0 |
06Sealed validation, including a negative
The analytic machinery above would deserve limited trust on its own. The programme therefore validates against sealed known-answer counterfactuals: for three real held-out sessions, the saving at a given trigger was computed by the product's own forward simulation, frozen as a sealed known, and must be reproduced within a one-sided band (known−5, known+1) by any subsequent run. The band is asymmetric on purpose. The method may under-claim; it may barely over-claim.
| real session (actual spend) | saved @100k | @150k | @200k | vs never-compact (1M) |
|---|---|---|---|---|
| 78c10074 ($6,501) | 71.3% | 67.7% | 64.2% | 98.1%* |
| 4f56a5c0 ($5,928) | 71.0% | 67.4% | 63.9% | 97.3%* |
| 4458ae27 ($1,200) | 21.3% | 9.8% | −2.1% | 95.6%* |
All cells reproduced in-band (9/9 on the original endpoints; the 150k and never-compact endpoints sealed under the same discipline). *Never-compact figures are bounds, not prices, and carry that flag in the sealed rows.
Two readings. First, saving declines monotonically with later triggers on all three sessions, consistent with the curve's direction and with the optimum sitting below every swept point. Second, the negative: on the smallest session, compacting at 200k cost more than it saved. A method that flattered would have missed it. A negative that lands inside its sealed band is the strongest evidence in this table, because it demonstrates the instrument can say no.
07Per-tenant calibration
Every constant in this paper is somebody's variable. Tokens-per-turn differs by repository; interruption tolerance differs by team; the extractor choice moves Cevent by an order of magnitude ($0.14 for a small model, $1.62 for a larger one that, in our bench sweep, cut extraction losses from seven items to one). The operating parameters are therefore not constants in the product but measured values with provenance: a fleet-wide base derived from all sessions, overridden per repository from that repository's own sessions, overridden again by an operator pin, with every value carrying its source and evidence reference. The demonstration case is the negative above: the same session that loses money at a 200k trigger saves 21% at 100k and more below it. The wrong constant costs money. The measured profile earns it.
This also fixes who the numbers belong to. The telemetry that feeds calibration is usage metadata only (counts, sizes, timings, curves), never content, enforced structurally by a store with no free-text columns; cross-tenant aggregates publish only above a minimum-tenant threshold, and below it the report is a named refusal rather than an empty chart.
08Related work
[1] Koherium, What Survives Compaction, 2026 — the fidelity counterpart: what the managed ceiling must preserve.
[2] Jeffrey, Graph-Guided Context Reduction for AI Coding Agents, Cogmeta Research 2026 — input-side context economics; independently states the compounding cache-creation mechanism.
[3] Jeffrey, Bug-Class Forcing Functions, Cogmeta Research 2026 — the sibling notebook; structural enforcement at commit time.
[4] State of AI: An Empirical 100 Trillion Token Study, 2026 — prompt-size growth; the coding lane.
[5] Zhao et al., WildChat, 2024 and Zheng et al., LMSYS-Chat-1M, 2024 — consumer sessions never meet this economics.
[6] Anthropic pricing documentation — cache read/write rates, TTLs, long-context terms.
[7] Gemini 2.5 Pro pricing — 2× input beyond 200k.
[8] GPT-4.1 pricing — flat-rate one-million window; the comparison case.
[9] Packer et al., MemGPT, 2023 — context as managed memory; the architectural frame this economics funds.
[10] Kwon et al., vLLM / PagedAttention, 2023 and Zheng et al., SGLang / RadixAttention, 2023 — the serving-layer mechanics of prefix reuse that make the cached column of Figure 1 possible at all.
[11] Chen et al., FrugalGPT, 2023 — LLM cost reduction by cascaded routing; a complementary lever operating per-query where this paper operates per-session.
[12] Zhang et al., H2O, Xiao et al., StreamingLLM, Li et al., SnapKV — KV-cache eviction under memory pressure; the closed-form trigger of §05 is the application-layer sibling of their token-layer eviction policies.
09Limitations
Bounds, not measurements, in Figure 1. The cached column assumes perfect prefix caching; the uncached column assumes none. Real sessions live between them, and the per-tenant telemetry (cache read and write tokens per real request) exists precisely to replace these bounds with measured hit rates. Until it does, every Figure 1 ratio is quoted against the bound most favourable to the unmanaged case.
Estate shape. The census is agent-heavy and coding-heavy, which is the population where the economics bite; it is not a claim about consumer chat, which public data shows rarely exceeds a few turns[5].
The look-back proxy. Reference distance is measured on file-path re-mentions. It is a behavioural proxy for retrospective need, not a complete account of it; knowledge re-derived without re-mention is invisible to it.
Single pricing regime. Dollar figures are computed at pinned August-2026 rates for one vendor. Rates change; the closed form does not, and re-deriving T* under new rates is arithmetic.
The wall-time term is real and unpriced. The dollar optimum at 28 compactions per session is not deployable interactively today. We state the practical optimum as cost plus a tenant-weighted interruption term rather than pretending the dollar curve is the whole objective.
10Claims and non-claims
- Session input-spend is quadratic in length and survives both caching and million-token windows; on measured shapes the unmanaged-versus-managed gap runs 9–30× at the caching bound most favourable to the incumbent.
- 81% of a session's retrospective references fall within its last 50 turns, about 33 KB; the reinstatement working set is small and measurable.
- The optimal compaction trigger has a closed form in three measurable inputs and lands well below common defaults; sealed sweeps on three real sessions are directionally consistent with it.
- Trigger timing is per-session: one real session loses money at a 200k trigger and saves 21% at 100k, reproduced in-band.
- That the Figure 1 dollar figures are prices; the cached column is a floor and figures above one million tokens are bounds, flagged as such wherever sealed.
- That the dollar-optimal trigger is the deployable trigger; wall-time per compaction is a real term and is stated, not solved.
- That the saving percentages transfer to any workload unmeasured; the calibration system exists because they do not.
- That a managed ceiling is safe without the fidelity results of the companion paper[1]; economics and preservation are one product claim, and the preservation half is still being earned on real text.
11Reproducibility
The census requires only session transcripts and a tokeniser approximation. The quadratic table is arithmetic from published rates and stated assumptions. The look-back study is mechanical given transcripts. The sealed counterfactuals are the strongest leg: knowns frozen before reproduction, a one-sided band, and every subsequent run required to land inside it, with the S150k endpoint added under a non-vacuity control (existing endpoints byte-identical through the change). An assessor implementing the same method against a reader's own sessions reproduces the entire chain on their data, which is where these numbers should be trusted, and the only place they should be sold.