CliffCompaction Cuts Coding Agent Token Costs by 50% Without Summaries
A new autocompaction proxy trims coding agent histories in place, cutting token bills roughly in half while lifting Terminal-Bench and KernelBench scores.
- New paper introduces CliffCompaction, an autocompaction proxy for long-horizon coding agents.
- Cuts token costs up to 50% while matching or beating full-context performance on Terminal-Bench.
- Only truncates or drops text, never rephrases, avoiding drift from summarizing summaries.
- Preserves SWE-bench Verified scores at 32K and 16K context thresholds for GLM-5.1 and Kimi K2.6.
- Hits 3.58x CUDA kernel speedups on KernelBench after 400 steps, beating specialized search pipelines.
- Ships as MIT-licensed API proxy on GitHub and PyPI, compatible with Claude Code and Codex.
Long-running coding agents repeatedly resend their conversation history, tool output, and reasoning with each model call. After hundreds of calls, that transcript can exceed the model’s context window, raise inference costs, and bury useful state under stale logs. Researchers Trang Nguyen, Eulrang Cho, Bingqing Chen, and Tim Dettmers propose CliffCompaction, a proxy that keeps each request within a fixed token budget by truncating or deleting text while leaving retained content unchanged.
The researchers report sessions spanning millions of cumulative tokens, token-cost reductions of about 50%, and benchmark results comparable to full-context runs at much smaller context thresholds. Their implementation supports Claude Code and Codex CLI, is licensed under MIT, and is available as a Python package.
| Property | CliffCompaction’s approach |
|---|---|
| Compaction method | Truncates or removes existing text; retained spans remain verbatim |
| Protected content | System instructions, task description, and recent turn pairs |
| Integration point | HTTP proxy between the agent client and model API |
| Supported APIs | Anthropic Messages, OpenAI Chat Completions, and OpenAI Responses |
| Configuration | Separate character budgets for tool results, assistant text, and reasoning blocks |
| Testing aid | Shadow mode records proposed cuts without changing requests |
Summaries accumulate errors
Coding agents can generate millions of tokens across a task as they inspect repositories, run tests, edit files, and recover from failed attempts. The model receives much of that history again on every call, so a growing transcript increases both input-token charges and the amount of irrelevant material competing for attention.
Many agent systems control that growth by asking a model to summarize older turns. Each summary can omit a constraint, alter a filename, blur the cause of a test failure, or discard an abandoned approach that should remain abandoned. Repeatedly summarizing earlier summaries compounds those changes during long sessions.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
