cost-saving · layer 2

Don't pay for the same bytes twice

Most of what a coding session sends the model is text the model has already seen: the same instructions, the same tool descriptions, the same file read twice. Squeezy stops paying full price for those repeats. It replaces output the model already has with a short reference, keeps the unchanging front of every request byte-for-byte identical so providers can serve it from their cache at a discount, and clears out lines an edit has made wrong.

contents

Where the repetition comes from

A model has no memory between turns. Every turn re-sends everything it needs to keep going: the standing instructions, the descriptions of the tools it can call, the whole conversation so far, and every file it has read. After a handful of reads and edits, that replayed history is most of what you pay for on each new turn. Your latest message is a rounding error beside it.

Providers charge for input by the token, and they do not discount text just because they have seen it before. Left alone, a long session re-buys its entire history every turn. This page is the four things Squeezy does to stop that.

Prompt caching: pay full price once, a fraction after

Providers offer a cache. If the beginning of your request is byte-for-byte identical to a recent one, they serve that part from a warm cache at a steep discount instead of the full input rate. The catch is that it only works on a prefix: the match starts at the very first byte and runs forward until the first byte that differs. Change one character early in the request and everything after it is charged at full price again.

Squeezy is built to keep that prefix from moving. The instructions and the tool list are assembled in the same order every step, so their bytes never shift. The facts that genuinely change every step go to the very back of the request, behind the point where the reusable part ends: the working checklist, the budget readings, the list of files an edit just made stale. Those can move every turn without disturbing anything in front of them. New conversation is only ever appended. So the long, unchanging front of the request stays cacheable turn after turn.

The payoff is real, and it is worth being clear about whose it is. The discount belongs to the provider. Any client that sends a stable prefix to a model priced for prompt caching earns it, which makes collecting it table stakes rather than a Squeezy invention. Where a provider does not price cached input, prefix stability still helps but there is no discount to collect. What Squeezy adds is the engineering that keeps the prefix stable enough for the discount to keep landing across a long session. That is also why /cost reports the cache beside your bill and keeps it out of the savings Squeezy claims for itself.

Fixed order: the instructions and the tool list are assembled the same way every step, so the cache lookup keeps succeeding.
Volatile facts at the back: per-turn readings sit behind the reusable part of the request, where changing them costs nothing.
Append-only history: each turn adds to the end, leaving the cacheable front untouched.
Longer sessions win more: the warm-up is paid once and reused by every later turn.

Tuning the cache, and seeing what it cost

Caching is a discount on reads and a surcharge on writes. Warming a longer-lived cache costs more up front, and only pays off if those bytes get read back enough times. Squeezy leaves this on automatic. It watches how long your turns actually pause between calls, and how much of the cached prefix is genuinely being re-read, and only reaches for the longer-lived, pricier window when the reuse is there to justify it. When the measurement says the prefix is going unread, it stays on the cheaper short-lived cache instead of paying for retention you would not use.

One setting overrides all of that. The cache mode takes four values: off, short, long, and auto. Off disables provider prompt caching outright, which is how you get a clean, cache-free cost baseline to measure against. Short and long pin the window. Auto is the default, and the behavior described above.

Then /cost tells you what the cache did to this session. It shows the share of your input that was served from cache, and the net dollar effect: the discount the reads earned, less the premium the writes cost. That figure is stated as a plain verdict, that the cache took money off your bill or added money to it, and it is labeled as not a Squeezy saving. Both halves of it are the provider's, so a cache that came out ahead never enters Squeezy's own savings total.

Automatic by default: the window is chosen from real turn timing and real reuse, so durable caching is not paid for when nothing is reading it.
One override: [cache].mode = "off" | "short" | "long" | "auto", with off giving you a cache-free baseline.
Trims that would not pay for themselves do not run: clearing old bytes out of a conversation makes the provider re-bill everything after the change, so Squeezy weighs the reclaim against that re-bill first.
Honest attribution: /cost states the cache's net effect on your bill and keeps it out of the savings Squeezy claims.

Receipts: a reference instead of the same bytes again

Agents are bursty re-readers. The model pulls up the same config file in turn 3 and again in turn 7, re-runs a search it already ran, or revisits the same page across one task. Each repeat would normally re-send the identical content and charge for it again, even though those exact bytes are already sitting earlier in the conversation.

So when a file read, a search, a symbol lookup, or a fetched page comes back byte-for-byte identical to one already sent, Squeezy does not send the content again. It sends a receipt instead: a short note naming the earlier step whose result this repeats. A read that would have cost thousands of bytes becomes a couple of hundred.

What keeps that safe is a rule about when a receipt is allowed. One is only ever sent while the original is still in the conversation for the model to look back at. If that earlier result has since been cleared out, Squeezy re-delivers the full content rather than hand over a pointer to nothing. Receipts are stored with the project too, so a session you resume picks up deduplicating where it left off instead of starting from zero.

A separate guard keeps one busy turn from flooding the conversation: total tool output per step is capped. If the model fires off a dozen large calls at once, what fits is sent in full and the rest come back as short stubs with a pointer for fetching them. The budget is spent in priority order, errors first, because they are small and are exactly what the model needs to make its next move, then results it explicitly asked to retrieve, then everything else smallest first.

Repeat reads, searches, and fetches: replaced with a short reference once identical content has already been sent.
Never a dangling pointer: if the original is gone from the conversation, the full content comes back instead of a reference.
Across a resume: receipts are stored with the project, so a resumed session keeps the deduplication it had built up.
Changed lines only: a re-read can be asked for as a diff against what you last saw, returning just the ranges that changed since.
Per-step cap: a burst of large results is bounded, errors are kept first, and anything held back carries a pointer for fetching it.

Deferred tool definitions: a compact index, details on demand

Every tool the model can call comes with a full description of how to call it, and those descriptions are sent on every turn, ahead of your conversation. Most turns touch two or three tools. A model reading a file and writing a patch does not need the full specification for fetching web pages, driving a browser, or inspecting background jobs, all of which are sitting in the request being paid for.

So Squeezy sends a menu instead. A small set of always-available tools keeps its full definition. Everything else is advertised as a single line: the tool's name, the kind of access it needs, and one line on what it does. When the model wants one of those tools it asks for the full definition, which then stays loaded for the rest of the session. Squeezy also attaches a definition on its own once the session reaches the state that needs it, so the tool for reading a background job's output shows up when you actually have one running.

This does double duty. It shrinks what goes on the wire from the very first turn, and because the menu is short and sorted the same way every step, it holds the front of the request still, which is exactly what the prompt cache needs. Skills follow the same pattern: the session carries a roster of the skills it found with one line each on what they are for, and a skill's full instructions are fetched only when the model decides to use it.

Compact menu: a deferred tool costs one line up front instead of its whole definition.
Load on use: the full definition arrives when the model reaches for the tool, and stays for the rest of the session.
Promoted when relevant: a definition is also attached automatically once the session enters the state that needs it.
Protects the cache: the menu is sorted the same way every step, so its bytes never churn.
Same idea for skills: names and one-line descriptions up front, full instructions on request.

Letting go of bytes an edit made wrong

Caching and receipts handle bytes that repeat unchanged. Editing creates a different kind of waste. A file you read earlier in the session is still sitting in the conversation at its old contents, and the moment you change that file those lines are both wrong and expensive, because every later turn re-sends them.

So when an edit lands, Squeezy goes back through the earlier reads and searches that touched that file and rewrites, in place, only the spans the edit invalidated. Each one becomes a short note saying what was cleared and where the current version lives. The most recent views of the file are left word for word, since those are the ones that informed the edit, and everything outside the changed span stays exactly as it was. It happens automatically, with no extra model call.

Only the changed span: for a search-and-replace, just the replaced text is cleared and the rest of that earlier read is untouched.
Only successful edits: an edit that failed, was denied, or was reverted never rewrites anything.
Recoverable: the note names the file, so the model can read it again if the cleared span still matters.

Why they compound

Each of the four attacks a different slice of the repeated cost, so they add up rather than overlap. Deferred definitions shrink the front of every request and hold it still. That steadiness is what lets the provider's cache discount the repeated prefix on every later turn. Receipts keep redundant reads and searches from ever entering the history the cache carries forward. And the stale-byte pass drops what an edit already made wrong.

The throughline is that recognizing “you have seen this before” is cheap, mechanical work your own machine can do. What you pay the model for is the new thinking in each turn, not a re-purchase of bytes you already bought. And every one of these steps keeps a route back to the original, so a saving never costs you an answer.

the contents list reaches every page in the docs