billion-context

billion-context is an open-source tool by GitHub user ranxianglei. The README says it sits between any agent and its model API, rewriting Anthropic and OpenAI streams with acp-kernel compression. The model can call compress, decompress, search-context and status tools on its own, and what's left after compression is a summary it can expand again. The model itself decides when to compress and when to pull details back, so older parts of the conversation aren't simply cut off. The project includes the local proxy and plugins for different clients, with a README in English and Chinese.

Who it's for:developers who run very long coding sessions and can read and check proxy configuration. Readers who don't write code won't get much from it.

Before you start: it can read and rewrite every request and response between you and the model, then forward them to the upstream API you configure. Sessions are stored in ~/.local/share/billion-context and logs in ~/.local/state/billion-context/bili.log. Some clients need you to trust its own certificate (a man-in-the-middle setup) and set HTTPS_PROXY; installing it for clients such as Claude Code adds a SessionStart hook and an MCP shell and rewrites the client configuration, keeping a .bili-bak backup. By default it listens only on 127.0.0.1; run in Docker, it listens on all interfaces with no authentication, and the project warns to use it only on trusted networks. It also checks for and installs updates automatically. We don't recommend opening outside listening, trusting the certificate, installing hooks or turning on auto-update without reviewing each one first. The code is free under the MIT licence; model API costs are separate. The author reports 5x savings and 95% to 97% cache hit rates from their own tests, which we haven't reproduced.

Based on the project's public documentation. Not tested by AI Inspiration World.