Don't pay for the same bytes twice

Where the repetition comes from

Every turn you take with a model isn't a fresh start. The model has no memory of its own, so each turn re-sends everything it needs to keep going: the standing instructions, the descriptions of the tools it can call, the full back-and-forth so far, and every file it has read. After a handful of reads and edits, that replayed history is the bulk of what you're paying for on each new turn, not your latest message.

Providers charge for input by the token, and they don't discount text just because they've seen it before. Left alone, a long session re-buys its entire history on every turn. Squeezy's job here is to recognize repetition and stop paying full freight for it, in several complementary ways.

Prompt caching: pay full price once, a fraction after

Providers offer a cache: if the beginning of your request is byte-for-byte identical to a recent one, they serve that part from a warm cache at a steep discount instead of the full input rate. The catch is that it only works on a stable prefix, the match has to start at the beginning and run forward until the first byte that differs. One changed character early in the request and everything after it is charged at full price again.

Squeezy is built around keeping that prefix stable so the cache keeps hitting. The standing instructions are fixed for the session, no timestamps, random IDs, or per-turn lines that would silently break the match. The tool list is ordered the same way every time. New content is only ever appended to the end, so the long, unchanging front of the request stays cacheable turn after turn.

The payoff is concrete: the first turn warms the cache, and every turn after reuses that warm prefix at a fraction of the price. Because coding sessions are dominated by repeated history, this is the single biggest lever Squeezy pulls, and it compounds in longer sessions, where the one-time warm-up is amortized over many turns.

• Stable instructions: the system context is held constant across the session so the cached prefix matches.
• Stable tool order: tools are listed deterministically so the cache lookup keeps succeeding.
• Append-only history: each turn adds to the end, leaving the cacheable front untouched.
• Longer sessions win more: the warm-up is paid once and reused by every later turn.

Tuning the cache, and seeing what it cost

Caching is a discount on reads but a small surcharge on writes: warming a longer-lived cache costs a little more up front, and only pays off if those bytes are read back enough times. By default Squeezy leaves this on automatic, it watches how long your turns actually pause and how much of the cached prefix is genuinely being re-read, and only reaches for the longer-lived, slightly pricier window when the reuse is there to justify it. If the prefix isn't being re-read, it stays on the cheaper short-lived cache rather than paying for durable retention you wouldn't use.

You can also set it by hand with a single cache mode: turn caching off entirely for a clean, cache-free cost baseline, pin the short or long window, or leave it on automatic (the default). And `/cost` shows a labeled estimate of the net effect, the discount the cache reads earned minus the premium the cache writes cost, so you can see whether caching is actually saving money on this session. It's an estimate, and it only calls the result a saving when the discount genuinely beat the premium, never the other way around.

• Automatic by default: the cache window is chosen from real reuse, and durable caching is not paid for when it isn't being used.
• Manual override: a single mode turns caching off for a clean baseline, or pins a short or long window.
• ROI you can check: /cost reports the net dollar effect as an estimate, and only claims a saving when reads outweighed writes.

Receipts: a reference instead of the same bytes again

Agents are bursty re-readers. The model will pull up the same config file in turn 3 and again in turn 7, re-run a search it already ran, or revisit the same docs page across one task. Each repeat would normally re-send the identical content and charge for it again, even though the model already has those exact bytes earlier in the conversation.

When Squeezy detects that a file read, search result, or fetched page is identical to one already sent, it doesn't resend the content. It sends a short receipt instead, a small note that says, in effect, “this is the same result you already got, from that earlier step.” A read that would have cost thousands of bytes becomes a couple hundred, and the receipt always points back to the exact earlier result, so the model can pull the full bytes back on demand if it needs them.

This reaches across sessions in the same project, too: open a fresh session tomorrow and the first read of a file you already looked at can come back as a receipt immediately. And to keep one busy turn from flooding the conversation, Squeezy caps total tool output per round, if the model fires off a dozen large searches at once, the ones that fit are sent in full and the rest come back as compact stubs with a pointer to retrieve them.

• Repeat reads and searches: replaced with a small reference once the content has already been sent.
• Recovery path: every receipt names the earlier result, and the full content can be pulled back on demand.
• Across sessions: a new session in the same project can get the first read of an already-seen file back as a receipt right away.
• Changed lines only: re-reading a file you have already seen can come back as just the lines that changed since, not the whole file again.
• Per-round cap: a burst of large results is bounded so one turn can't dump everything at once.

Deferred tool definitions: a compact index, details on demand

Each tool the model can call comes with a full description of how to call it, and those add up, a rich toolset can be tens of kilobytes of definitions sent on every turn, before any of your conversation. In a typical turn most of those tools go untouched: a model reading two files and writing a patch doesn't need the full spec for fetching web pages or every other capability just sitting there being paid for.

Squeezy sends a compact index instead, a short list of what tools exist, each a single line with its name and a one-line description. The model reads the menu, and only when it wants a tool does it ask for that tool's full definition, which then stays loaded for the rest of the session.

This does double duty. It shrinks what's on the wire from the first turn, and because the index is short and ordered consistently, it keeps the front-of-request prefix stable, exactly what the prompt cache needs to keep hitting. Skill instructions follow the same pattern, so a session can have many capabilities available while only paying in full for the ones it reaches for.

• Compact menu: tools are advertised by name and a one-line description, not their full definitions.
• Load on use: a tool's full details arrive only when the model decides to call it.
• Protects the cache: the short, stable index keeps the cacheable prefix from churning.
• Same idea for skills: instruction bodies are summarized up front and fetched in full only when needed.

Why these three work together

They compound because each targets a different slice of the repeated cost. Deferred definitions shrink the front of every request and keep it stable. That stability lets prompt caching discount the repeated prefix on every later turn. And receipts strip out the redundant reads and searches before they ever pile up in the history the cache carries forward.

The throughline is simple: the cheap, repetitive work of recognizing “you've seen this before” is handled locally, so your paid tokens go to the new reasoning of each turn instead of re-buying bytes you already paid for, with a recovery path always available so the savings never cost you correctness.

Letting go of bytes that went stale

Caching and receipts handle bytes that repeat unchanged. A different kind of waste appears the moment you edit: a file you read earlier in the session is still sitting in the history at its old contents, and those lines are now both wrong and expensive to keep re-sending on every later turn.

So when an edit lands, Squeezy goes back through the earlier reads and searches that touched that file and rewrites just the parts the edit invalidated, in place, into a short note that points at the current version. The model keeps an accurate picture without anyone re-reading the file, the freshest views are left untouched, and the stale bytes stop riding along in every future turn. It happens automatically, with no extra model call.

Related

• Cost-saving strategies
• Control the bill