Graph-Guided Context Reduction for AI Coding Agents: A Controlled Empirical Study

01Introduction

AI coding agents read files, run commands, and edit source inside an expanding context window. Watch one work and the pattern is hard to miss: most of what it does is not writing code. It is searching for the right code to change: grepping, reading, re-reading, verifying. Discovery is the hard job. The edit itself, once the agent knows where it is going, is usually short. And because the context window is finite, expensive and accumulative, every step of that search is paid for again on every subsequent turn.

Every file read, every command output, and every assistant response adds tokens to the conversation history. In long-running sessions — the kind required for non-trivial feature work — the context window fills with stale tool results, redundant file reads, and exploratory output that was useful at the time but contributes nothing to subsequent turns. Our analysis of 9,459 real-world agent traces (304,929 individual steps)^† found that 47.5% of all agent actions are either redundant (28.5%) or outright waste (19.0%). Only a third of steps directly contribute to task completion.

^† Traces collected from Cogmeta's internal A/B benchmark runs and anonymised early-alpha telemetry on claude-sonnet-4-6. Source code never recorded; only structural step metadata (tool kind, file path, token counts, durations). Corpus identifiers and per-source counts in §15.

The cost implications are significant. Anthropic's API pricing includes both input token costs and cache creation costs. Every token injected into the context window — whether the model uses it or not — incurs cache creation overhead. As context grows, costs compound quadratically — self-attention scales with the square of the sequence length (Vaswani et al., 2017; Beltagy et al., 2020), and the key-value cache that avoids recomputing it becomes the dominant memory cost of serving long contexts (Kwon et al., 2023): a 100-turn session doesn't cost 100× a single turn; it costs far more, because early turns are re-processed in every subsequent API call.

Talon does the discovery for the agent. It predicts the context each step needs, serves it before the agent goes searching, and steers the agent away from the exploration that inflates context windows. We publish the outcomes rather than the recipe. The sections below report sealed paired measurements, the cells where Talon loses, and a static way to measure (before you pay anything) how much your own codebase stands to save.

Related Work

Discovery cost sits in a field a decade into retrieval and two years into agents. We situate against it plainly, and we claim less than the neighbours where the neighbours claim more.

Retrieval-augmented generation (Lewis et al., 2020) set the shape everyone now inherits: fetch relevant context, put it in front of the model, let the model condition on it — and for code specifically, semantic natural-language-to-code search is a hard retrieval problem in its own right (Husain et al., 2019). Talon is a retrieval system in that lineage. What differs is the objective. Classical RAG asks whether a retrieved passage is relevant. We ask what the retrieval costs on every subsequent turn, because an agent re-pays for its context on each API call, and relevance alone does not bound that bill.

That serving less can beat serving more is not folklore. Liu et al. (2024), "Lost in the Middle", measured that models under-use information buried in a long context; retrieval into a short window can match or beat a much longer context at a fraction of the compute (Xu et al., 2024). The finding cuts straight against the naive graph dump: a large, accurate context is not a free good, and the tokens the model skims still cost cache-creation on write. Our cache-creation results in §07 are the economic shadow of that behavioural one.

The loop we optimise is the reason-and-act loop of ReAct (Yao et al., 2023) — the backbone of the modern coding agent (Xi et al., 2025), extended with chain-of-thought reasoning (Wei et al., 2022) and self-reflection across attempts (Shinn et al., 2023): the model thinks, calls a tool, reads the result, thinks again. Every one of those tool calls is a discovery step, and the efficiency floor in §03 is a statement about how few of them a task truly needs. It matters here that a model left to check its own work cannot reliably self-correct without an external signal (Huang et al., 2024) — the rediscovery-and-re-verification that inflates so many sessions is that failure mode at scale. We do not change the loop. We change what the first read returns.

The closest tooling family is repository-level retrieval for code. RepoCoder (Zhang et al., 2023) iterated retrieval against a repository for code completion, and repository-scale evaluation is now standardised by benchmarks such as RepoBench (Liu et al., 2024); the repo-map tooling in shipping agents (Aider's repo-map, Cursor's indexing) hands the model a structural sketch of the codebase up front. These target the problem we target. Where we differ is measurement. We publish sealed paired A/B cost per cell, the size floor where the approach loses money, and the losses, rather than one aggregate on a public set.

We measure broken-test rate and resolve on SWE-bench (Jimenez et al., 2024), the real-GitHub-issue benchmark, over a 100-task subset we call SWE-bench-100. It is the field's common ground for whether an agent actually fixed the issue. We report parity on it rather than a headline lift, because the contribution is cost at held quality, not a solve-rate claim.

Stated against that backdrop, the contribution is narrow. We take the discovery loop that RAG and ReAct describe, put a structural index in front of it in the RepoCoder tooling family, and measure the thing those lines mostly leave implicit: what the context costs, paired, sealed, with the cell where it costs us money left on the board. We do not claim a new retrieval mechanism. We claim an honest price.

02Discovery and Measurable Opacity

If discovery is where the cost goes, the saving should track how hard a codebase is to search. It does. The deepest sealed cells sit on large, private, heavily-indirected code: the 0.41× end of the Python range, the 35–50% band above 400K LOC. The saving shrinks as the repository gets easier to grep, and crosses into a published loss on a ~1K-file TypeScript repo an agent can hold in its head. The 47.5% waste figure from the trace corpus (§01) is the same fact seen from the other side: what agents waste is search, so what Talon saves is bounded by how much search the codebase demands.

That has a useful consequence. Search difficulty is a property of the repository, not of the task. You can measure it before running anything.

2.1 talon estimate — know the band before you pay a cent

talon estimate is a static scan: no LLM, no API key, roughly thirty seconds on a working checkout. It reads repository structure only and returns a savings band: an estimate of where your codebase sits between the 1.236× small-repo loss and the deepest sealed cells.

We state its standing precisely. The scan is a screen, not a predictor. Its output correlates with the sealed per-cell savings at rank-correlation level, and the anchor set behind that correlation is the sealed cells themselves, a small set rather than hundreds of calibration points. It will tell you whether your repository looks like the kind where Talon saves 40% or the kind where it costs you money. It will not tell you your invoice. We publish the band and what it correlates with; we do not publish the formula.

03The Efficiency Floor

For any task there is a computed floor: the smallest number of tool calls an agent would need if every call landed. The floor is the honest benchmark for a context engine, because it separates "the agent worked less" from "the agent worked the minimum".

Talon's tuned serves reach it. On the floor benchmark the Talon arm completes in 4 tool calls where an unassisted agent takes 14; on a real spreadsheet task the gap was 3 against 15. The other ten calls are not mysterious. Roughly 60% of the gap is rediscovery and verification: the agent re-finding what it already found and re-checking what it already checked.

The verification share is the interesting part. Models are trained to check their work, and an agent that never verified would be a worse agent. Verification is constitutional. Some of it survives even a perfect serve, which is why the floor is four calls and not one. What Talon removes is the rediscovery, and the verification spirals that rediscovery feeds.

We state the floor in tool calls and turns, deliberately. Tokens per call vary with the task, so a token-denominated floor would be an estimate dressed as a measurement, and we don't publish those.

04Methodology

The protocol below is the marathon-era design used by the superseded corpora in §05–§06 and retained for the record. The sealed cells in the abstract use the same paired A/B discipline on single-task vehicles at K=2, with resolve checked in both arms.

4.0 How to read these numbers

The sealed cells report paired A/B effect sizes and per-task win counts, not a single population statistic. Read a cell like Java as "0.605× cost, 11/12 wins, resolve 7/12 versus 7/12": the ratio is the paired cost, the win count is how many of the twelve tasks Talon ran cheaper, and the resolve line is the parity check. We treat those as descriptive, paired measurements of the same task run both ways.

Where a cell has the samples to carry an interval, we give one. Solve rate sits at roughly 60% in both arms and the gap is inside the noise, so we call it a statistical tie rather than a lift. The paired-judge quality scores in §09 average three rounds precisely to pull per-call variance down from about 17 points to about 5. Where a cell is small (the unfamiliar-code probe at n=23, the beyond-code probes at n=2–3) we say so, and we hold it to descriptive, paired reporting only. We do not attach a p-value or a confidence interval to a cell that cannot support one. An invented interval would read as rigor and be the opposite.

So the rigor here is not a significance star. It is the sealed standard, the paired design, and the published losses: the same task, run both ways, with the cell where we cost you money left on the board.

4.1 Experimental design

All experiments follow a controlled A/B protocol. Each task is executed twice: once as a vanilla run (the agent with no external context assistance) and once with Talon active. Both arms use the same model (Claude Sonnet), the same Claude Code binary, and identical task prompts. Tasks are multi-prompt marathons, each comprising 9–13 sequential prompts that build on each other to complete coherent work — feature additions, refactors, or integrations into real open-source codebases.

We chose marathon-format tasks rather than single-prompt benchmarks because they expose the compounding effects of context accumulation. A single-prompt task doesn't produce context bloat. The pathologies we study — redundant reads, cache thrashing, scope drift — only emerge over sustained multi-turn sessions, which is how professional developers actually use these tools.

4.2 Metrics

All metrics are extracted directly from the Claude API response JSON, specifically the total_cost_usd and num_turns fields. We do not parse conversation transcripts to estimate token counts, as transcript-based methods systematically overcount by ~1.5× due to encoding differences. Primary metrics are:

• API cost (USD) — total billed cost, including input tokens, output tokens, and cache creation/read.
• Conversation turns — number of assistant turns required to complete the task.
• Total context tokens — sum of all input tokens processed across the session.

4.3 Task selection

The RC2 benchmark comprises 12 tasks split evenly between greenfield projects (built from scratch) and existing repos (feature additions on established Python projects: APScheduler, Celery, Dramatiq, Redis Streams, RQ, Scrapy). Greenfield tests whether Talon helps even with no pre-existing codebase; existing-repo tests performance on mature, structurally complex projects.

4.4 Exclusions

One task from the original RC2 candidate pool of 13 was excluded due to API quota exhaustion during the vanilla arm. All exclusions are reported. We do not selectively remove tasks to improve aggregates — as evidenced by the two tasks (evflow-zero, fsm-lib) where Talon increased costs, which are included in all reported numbers.

05Results · RC2 Benchmark (12 tasks)

⚠ SUPERSEDED — DO NOT CITE. The RC2/TREF1 marathon aggregates below predate our clean re-validation. Re-run on a clean vehicle, the marathon format measured a cost increase (1.60×), so these aggregates are retained for the record, not as claims. Current claims are the sealed cells and ladder results in the abstract. The per-task negative results remain valid and corroborated.

The RC2 benchmark was our broadest early experiment, covering 12 independent marathon tasks across both greenfield and existing-repo conditions.

5.1 Aggregates by segment

The cost saving appears on code the model doesn't already know, private or post-cutoff repositories, where the structural serve replaces blind exploration. On well-known public frameworks the model already carries the map, and the serve is redundant. The discriminator is model-familiarity, not repo-maturity.

5.2 Per-task · greenfield

Greenfield tasks were built from scratch in temporary directories with no prior code. These represent the lower bound of Talon's effectiveness — there is no existing structure to draw on at session start. Despite this, five of six tasks showed cost reductions, and all six showed turn reductions.

Negative result Two small greenfield tasks (evflow-zero, fsm-lib) showed cost increases of +37% and +12%. Both had vanilla baselines under $3. At this scale, the fixed overhead of graph indexing and context injection doesn't amortise. Turns still decreased in both cases (−27%, −35%), suggesting the agent is more decisive per-turn even when total cost increases. This establishes a clear lower bound: Talon's cost benefits require tasks of sufficient complexity.

5.3 Per-task · existing repositories

Existing-repo tasks were feature additions and refactors on established open-source Python projects, Talon's intended operating environment. Performance improvements were consistently stronger than in the greenfield condition, with the two best results (dramatiq-circuit −66%, scrapy-middleware −65%) occurring on projects with the deepest structural complexity.

Interesting outlier rq-observability consumed 103% more context tokens with Talon active, yet still achieved a 40% cost reduction. The agent loaded more context per step but required far fewer steps overall. This demonstrates that context efficiency (right tokens at the right time) matters more than context volume — consistent with the scoring hypothesis.

5.3.1 Distribution of per-task savings

Reading the curve below: pick a savings level on the x-axis and the y-axis tells you what fraction of the corpus reaches that level or better. The curve crosses 50% at −47% (median) and 32% at −60% (top third).

5.4 Per-task results · TREF1 multi-prompt (6 tasks)

The TREF1 sub-corpus is six multi-prompt fseries marathons — same task names as the GA209 single-prompt sweep in §06 below, but each task is a 9–10 prompt sequence carrying state forward. The compounding effect of context reduction across prompts produces materially larger savings than the single-prompt RC2 segments. Vanilla baseline aggregates 3 vanilla runs per task on most tasks; Talon side aggregates 5–6 runs per task.

Note · turns drift Three TREF1 tasks (django-query-timeout, airflow-priority, sqla-pool-metrics) show more turns under Talon. The agent runs longer because context efficiency lets it complete the work properly rather than truncating; the cost saving comes from each turn being dramatically cheaper, not from fewer turns. This is the inverse of the RC2 single-prompt pattern, where turn reduction dominates.

06Results · GA209 Marathon (6 tasks)

⚠ SUPERSEDED — DO NOT CITE. The clean-stack re-measure of this regime ran +1.44× total context — the headline context-reduction figure from this era does not survive re-validation and is withdrawn.

The GA209 experiment targeted a different question: how does Talon perform on complex, multi-file tasks that push sessions to their limits? These tasks were chosen to be more challenging than RC2, requiring deep cross-file reasoning, multi-module refactors, and sustained context over 50+ turns. This represents the upper bound of Talon's impact — conditions where context accumulation is most severe and where scoring has the most to remove.

The cache creation reduction (−59.9%) is particularly significant. Cache creation costs are incurred for every token injected into context, regardless of whether the model attends to it. By reducing the volume of injected tokens through scoring, Talon directly reduces this fixed overhead. This is the mechanism that distinguishes scored context injection from naive graph dumps, which increase cache creation costs even when the graph data itself is accurate.

6.1 Relationship to TREF1 multi-prompt

The 6 task names exercised here (airflow-priority, celery-compression, celery-worker-hooks, django-query-timeout, django-tracing, sqla-pool-metrics) are the same surface tested under multi-prompt fseries conditions in §5.4 TREF1. The GA209 single-prompt sweep above measures the upper bound of context-reduction in isolation; the TREF1 table reports the matched paired multi-prompt cost results.

07The Cache-Creation Mechanism

The mechanism by which scored context injection cuts spend is the reduction in cache-creation tokens — the input tokens billed at the cache-write tier. In the marathon-era corpora (§03/§04, since superseded) this signal was the largest and most reliable of the three primary metrics; the mechanism itself carries into the sealed cells, where it is measured per-task rather than as a marathon aggregate.

This metric is informative because cache-creation cost is incurred for every token injected into the context regardless of whether the model attends to it. By surfacing fewer, more relevant files through structural-graph scoring, Talon reduces this fixed overhead directly. This is the mechanism that distinguishes scored context injection from naive graph dumps, which increase cache-creation costs even when the graph data itself is accurate. Downstream cost effects compound as fewer cached tokens are re-billed across prompts.

Illustrative, superseded corpus. The compounding pattern is clearest on TREF1, whose marathon aggregates are superseded (§05). Each fseries marathon replays a 9–10 prompt sequence carrying state forward, and each prompt that injects fewer cache-creation tokens than vanilla saves not only its own cost but also the cost of those tokens being re-cached on every subsequent prompt. On that superseded corpus the reduction widened as the marathon progressed: the largest TREF1 task, django-tracing, measured a −82% context-token reduction. We show it for the mechanism, not as a current figure.

08Context Composition

Illustrative, superseded corpus. The per-turn breakdown below is drawn from a single marathon session (celery-worker-hooks-001) in the superseded marathon regime of §05–§06. The composition mechanism it shows carries into the sealed cells; the session deltas cited here (−34% cost, −45% total tokens) are marathon-era figures, retained to illustrate where the reductions land, not as current cost claims.

To understand where context reductions occur, we performed a detailed per-turn token breakdown on matched sessions from a 10-prompt marathon (celery-worker-hooks-001). The charts below decompose every turn into its constituent token categories — system prompt and tool definitions, user prompts, tool results, and assistant output.

Session comparison. Peak context −38K (166K → 128K). Zero compaction events vs one. Cost −34% ($12.15 → $11.09). Total tokens −45% (17.1M → 9.4M). The Talon session maintained context below the compaction threshold for the entire duration, eliminating the information loss that occurs when context is forcibly truncated — the knowledge loss measured directly in our companion research note, What Survives Compaction.

09Agent Behaviour Quality

To quantify the baseline waste in AI coding agent behaviour, we developed an automated quality scoring system and applied it to 9,459 real-world agent traces comprising 304,929 individual steps. Each step was classified into one of four categories. This analysis was conducted independently of the A/B experiments.

The key finding is that nearly half of all agent steps (47.5%) fall into redundant or waste categories. Redundant steps re-read files already in context, re-discover symbols that could have been located with a single structural query, or re-execute commands whose output is already available — exactly the behaviours scored context eliminates. Waste steps are out-of-scope exploration, test-fix-test spirals, and edits to unrelated files — exactly the behaviours guardrails prevent.

This analysis provides the theoretical ceiling for context-guided optimisation: if all redundant and waste steps could be eliminated, agent sessions would require roughly half the turns and substantially less than half the tokens.

9.1 Quality vs vanilla · paired-judge consensus

The TREF1 and TREF1-RC2 runs were designed and executed to measure cost and operational efficiency; output quality was not an optimisation target during the runs themselves. The paired-judge quality measure presented below was devised and applied after the runs, against the preserved workspaces. This is the first time we have measured task-level output quality alongside the cost and context metrics.

Headline: an independent paired-judge consensus rates Talon at parity or better on half the comparable tasks (9 of 18), with the remainder concentrated in low-cost or integration-heavy tasks where vanilla's diff shape is preferred, at materially higher cost.

Cost deltas in this section are superseded-corpus. The Δ cost figures shown alongside these quality scores (the "Δ cost vs vanilla" column of Table 7, and the −47% to −64% cited below) are from the superseded marathon corpus of §05–§06. They are shown to pair cost against quality on the same runs, not as current cost claims. Current cost is the sealed board in the abstract.

Verdict & severity mapping

Per-task quality scores

Half the corpus (9 of 18 tasks) scores at parity or better; one third (7) scores in the A−–A+ band. The lower half clusters on integration-heavy tasks where vanilla's tighter coupling to the project's existing extension patterns produces a diff shape the judge actively prefers — though Talon still completes the work at materially lower cost on most of those (−47% to −64% on four of the six lowest-scoring tasks). No task scored below 3.0; there are no decisive vanilla wins.

Methodology. Single Opus call per (task, round) using each side's git-diff text. The judge sees the task description plus both diffs (truncated at 8 KB per side; tooling/cache/agent-config paths and binary diffs filtered before truncation) and returns a verdict, severity, and per-axis subscores (correctness, completeness, focus, code quality). Three rounds reduce per-call variance from ~17pp to ~5pp. The 0–10 score is the mean across rounds of the verdict-and-severity rubric above; the per-axis subscores are reported in the supporting dataset.

Scope & framing. A reasonable hypothesis for future iteration is that a configuration jointly optimised for cost and quality — for example, richer neighbour-pattern context on integration-heavy tasks alongside the cost-saving discipline already in place — would lift the lower-grade scores without materially eroding the cost savings, since the structural-graph mechanism driving the cost wins is largely orthogonal to the diff-shape preferences the judge currently penalises.

10Freshness Economics

A context engine is only as good as its index, and an index goes stale the moment you commit. We measured our own weakness here rather than assuming it away. A graph left five commits stale misses 62% of the symbols in changed files. And changed files are exactly the ones the next prompt is most likely to touch, precisely because they just changed.

The fix is measured too. Incremental reindexing closes the gap at roughly 2.7 seconds per commit, so the graph tracks the working tree rather than trailing it. The 62% figure is what the product would quietly cost you without that loop; we publish it because a staleness failure is invisible from the outside. The serve still arrives. It is just wrong.

11Beyond Code — a Research Direction

This section is a research direction, not a product claim. Talon's launch identity is the context layer for AI coding agents. Nothing below is a shipped feature.

Nothing in the discovery mechanism is specific to source code. Any structured artefact an agent must search (a workbook, a document tree, a linked data set) has the same shape: the expensive part is finding the right cells to change, and the agent pays for the search on every turn.

We ran small constructed probes. On a real 3-statement financial model, the assisted arm cost 69.6% less and used 80% fewer tool calls, with output verified identical to the unassisted arm. It also caught something the unassisted agent did not: a completeness bug the unassisted arm shipped with a confident "verified". On an HTML document task the saving was 31%; on linked-JSON tasks, 49% rising to 62%.

The caveats are visible and they are large. These are n=2–3 constructed tasks, not a benchmark. The code cells above are sealed; these are not. We publish them as a direction: discovery cost appears wherever agents search structured artefacts, and the mechanism that measures and removes it in code appears to travel.

12Where Talon Doesn't Help

A serving engine can fail in both directions. We publish both. This register is part of the product, not an apology for it.

• Over-serving and mis-serving. Talon can serve context a step did not need, or serve the wrong context for the step. Both cost real money, and both are why the parity checks exist in every sealed cell. A cheaper run that resolves less is not a saving.
• NO_HEADROOM verdicts. Some tasks return a NO_HEADROOM verdict: the unassisted agent is already near the efficiency floor and there is nothing to save. Running Talon there buys you indexing overhead and nothing else.
• Small, greppable repositories. The ~1K-file TypeScript cell costs 1.236×, the published loss on the sealed board. Below the size floor, the agent's own grep is cheaper than any serve.
• Famous public code. On heavily-published frameworks the model already carries the map, and the serve is redundant. Measured cost: +9–19%.

talon estimate exists so that most of these cases are visible before you spend anything. Where the scan puts you near the floor, the honest advice is not to buy.

13Key Findings

14Limitations and Caveats

We believe transparency about the limitations of our methodology strengthens rather than weakens the evidence.

Single-model evaluation. All experiments were conducted using Claude Sonnet. While the underlying mechanisms (context accumulation, cache creation, redundant reads) are model-agnostic, magnitudes may differ across models with different context windows, pricing, and reasoning capabilities. We plan to extend to additional models.

Prompt sensitivity. Both arms use identical prompts, but stochastic agent behaviour means no two runs are exactly alike. We mitigate this with sample size (12+ tasks) rather than repeated trials of individual tasks — prioritising breadth over depth of repetition.

Negative results included. Two of twelve RC2 tasks showed cost increases. We report these in all aggregates rather than excluding them. The conditions that produce negative results are well-characterised and represent a known boundary.

Python-heavy test corpus. The existing-repo tasks in RC2 are all Python projects. While the parser supports 12+ languages, A/B data for non-Python is currently limited. Polyglot codebases may produce different distributions.

15Discussion

The central question motivating this research is whether doing the agent's discovery for it meaningfully reduces operational cost. The sealed cells answer a qualified yes. On codebases that are hard to search (large, private, heavily indirected) paired cost sits at 0.41–0.76× with solve parity, and the saving shrinks in a measured way as the codebase gets easier to grep, crossing into loss on small repositories. The discriminator is search difficulty. That is why it can be screened from the repository alone, before any agent runs.

The mechanism deserves emphasis. Talon does not make the model smarter, faster, or more capable. It provides the model with better starting information (the right files, not all the files) and constrains it from behaviours that waste context. The model does the same work — it simply does less unnecessary work.

The cache creation finding has implications beyond this specific product. Any tool that injects context into an LLM prompt — RAG retrieval, repo-map generation, custom system prompts — should account for the cache creation cost of that injection. Injecting 50K tokens of "possibly relevant" code creates 50K tokens of cache cost even if the model ignores most of them. This is the fundamental difference between scored and unscored context injection.

The negative results on small tasks are not a failure of the approach but a characterisation of its operating envelope. Every instrumentation layer has overhead. Talon's overhead is graph index maintenance, scoring computation, and context injection itself. For tasks completing in under 10 turns at less than $3, this overhead exceeds the savings. For the tasks that constitute the bulk of real engineering work, the return is substantial.

16Reproducibility

Every cell runs on a public repository. The quality benchmark is SWE-bench-100 — a subset of the public SWE-bench GitHub-issue set — and the named cost cells (django-tracing, celery-compression, sqla-pool-metrics, and the rest) are ordinary open-source projects. The paired A/B is reproducible on the same code: install Talon, run the identical prompt with it off then on, and compare. The measurement rests on public code, not on data only we can see.

We encourage users to verify these results against their own codebases. Talon includes a built-in session scoring command that applies the same quality classification used in §09 to your own sessions. After any task with Claude Code, running talon score-session produces a per-session breakdown of essential, useful, redundant, and waste steps — with token costs attributed to each category. This allows direct comparison between Talon-assisted sessions and your vanilla baseline.

$ talon score-session ~/.claude/sessions/47e2f1.jsonl

session       47e2f1 · 14 turns · $4.32 · sonnet-4-6
─────────────────────────────────────────────────────
category      steps   tokens     cost      share
─────────────────────────────────────────────────────
essential        5    18,420   $1.42       33%
useful           3    12,180   $0.94       22%
redundant        4    19,640   $1.51       35%   ⚠
waste            2     5,760   $0.45       10%   ⚠
─────────────────────────────────────────────────────
totals          14    56,000   $4.32      100%

eliminable     6 steps · $1.96  (45% of session cost)
top pattern    P-01 · consumer.py · re-read 6×
projection     Talon would have surfaced the file once ·
               cost ≈ $2.36 · turns ≈ 9
─────────────────────────────────────────────────────
write report  /tmp/talon-47e2f1.html        ▸ open

The scoring dashboard at app.cogmeta.ai aggregates these metrics over time, providing visibility into context efficiency across projects and team members. We believe the most compelling evidence is evidence you can reproduce on your own work.

References

1. Vaswani, A., Shazeer, N., et al. Attention Is All You Need. NeurIPS 2017. arXiv:1706.03762
2. Beltagy, I., Peters, M. E., Cohan, A. Longformer: The Long-Document Transformer. 2020. arXiv:2004.05150
3. Kwon, W., Li, Z., et al. Efficient Memory Management for Large Language Model Serving with PagedAttention. SOSP 2023. arXiv:2309.06180
4. Liu, N. F., Lin, K., et al. Lost in the Middle: How Language Models Use Long Contexts. TACL, vol. 12, 2024. arXiv:2307.03172
5. Lewis, P., Perez, E., et al. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. NeurIPS 2020. arXiv:2005.11401
6. Xu, P., Ping, W., et al. Retrieval meets Long Context Large Language Models. ICLR 2024. arXiv:2310.03025
7. Yao, S., Zhao, J., et al. ReAct: Synergizing Reasoning and Acting in Language Models. ICLR 2023. arXiv:2210.03629
8. Shinn, N., Cassano, F., et al. Reflexion: Language Agents with Verbal Reinforcement Learning. NeurIPS 2023. arXiv:2303.11366
9. Huang, J., Chen, X., et al. Large Language Models Cannot Self-Correct Reasoning Yet. ICLR 2024. arXiv:2310.01798
10. Xi, Z., Chen, W., et al. The Rise and Potential of Large Language Model Based Agents: A Survey. Science China Information Sciences, 2025. arXiv:2309.07864
11. Wei, J., Wang, X., et al. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. NeurIPS 2022. arXiv:2201.11903
12. Zhang, F., Chen, B., et al. RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation. EMNLP 2023. arXiv:2303.12570
13. Liu, T., Xu, C., McAuley, J. RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems. ICLR 2024. arXiv:2306.03091
14. Husain, H., Wu, H.-H., et al. CodeSearchNet Challenge: Evaluating the State of Semantic Code Search. 2019. arXiv:1909.09436
15. Jimenez, C. E., Yang, J., et al. SWE-bench: Can Language Models Resolve Real-World GitHub Issues? ICLR 2024. arXiv:2310.06770