Headroom: Compressing Agent Context Without Throwing Away the Original

An agent can spend a surprising amount of its context window reading repetitive tool output: search hits, JSON records, and logs that accompany the one line it needs. Headroom is an Apache 2.0 project that compresses selected context locally before an application sends it to a model. It offers a Python library and a local proxy for existing clients.

The interesting part is the recovery path. Headroom can cache the original output and offer a retrieval tool when a compressed result is insufficient. That makes it worth examining for agent workflows with bulky, repetitive results. It also introduces a local data store and another component in the request path, so token savings alone are not enough to judge it.

What gets compressed

The project README describes a content router that chooses different strategies for structured data, code, and text. Its SmartCrusher handles JSON arrays, retaining selected items such as errors and statistical outliers. The goal is to send a representative, smaller result to the model instead of repeating every row.

The maintainers’ own four example agent scenarios report input-token reductions from 21% to 57%. Those are project benchmarks, not measurements of your workload or a guarantee that answers remain equivalent. Repetitive records have more room to shrink than already dense prose. Compare the actual task outcomes and provider billing on your own representative inputs before relying on a percentage.

Compression, cache, retrieval

In the documented Compress-Cache-Retrieve (CCR) design, Headroom stores the original content with a hash and adds a retrieval marker to the compressed result. On supported proxy paths, a model can request the original through headroom_retrieve. Retrieval brings the missing material back into the conversation, so it can also add tokens and another step to a request.

This is recovery on demand, not a promise that the model will always ask for the right missing detail. The entry must still exist, and the client and provider path must support the retrieval flow. The documentation says the proxy defaults to a SQLite ccr_store.db in the workspace, with time-based retention and eviction when capacity is reached. Treat that file as potentially sensitive: its purpose is to hold original tool output. Review its location, access, retention and backup policy before using real data.

Try the library with a controlled input

For an application that already constructs model messages, the README documents a Python entry point. It requires Python 3.11 or newer. In a disposable virtual environment:

python -m pip install headroom-ai

Then pass your existing message list to the compressor before the provider call:

from headroom import compress

# Use the same message dictionaries your application would send to the model.
result = compress(messages, model="gpt-4o")
print(result.tokens_saved, result.compression_ratio)
# Send result.messages through your existing provider client.

messages is your application’s existing list; this excerpt is a starting point rather than a standalone script. According to the project, tool and assistant messages are compressed by default while user messages are left alone unless explicitly enabled. Inspect result.messages with non-sensitive test data, then compare answers against the uncompressed baseline. These commands and behavior are documented by the maintainers; they were not executed for this article.

For an existing coding agent, Headroom also documents a session-scoped wrapper and proxy. Review the installation and agent instructions before routing credentials or traffic through an extra local process. In particular, check the upstream URL for third-party OpenAI-compatible providers; the README warns that the default upstream otherwise points at OpenAI.

Measure the tradeoff, not just the tokens

Start with a small set of real task shapes using sanitized data: a long log with an error in the middle, a search result where the relevant hit is not first, and a structured response with an unusual field. Compare the answers, missing details, input and output tokens, and latency with compression disabled and enabled. Include cases where retrieval is needed, and test what happens after cache expiry or eviction. A lower input count is useful only if the task still succeeds at an acceptable total cost.

The README says an anonymous telemetry beacon is enabled by default and documents HEADROOM_BEACON=off and HEADROOM_OFFLINE=1 as opt-outs. Read the current privacy and proxy documentation for your configuration. Local compression does not make the upstream model call local: the resulting prompt still goes to the provider you configure.

Headroom is most compelling when repeated tool output dominates the prompt and a team can test its own failure cases. For short prompts, the extra process and local retention may buy little. Start with the library or an isolated proxy experiment, keep the uncompressed baseline, and adopt it only where the observed results justify the added moving parts.