benchmarks

show me the numbers, not the adjectives.

The saving, measured.

Two boards put squeezy against a baseline, an external coding-agent CLI, on the same 15 questions: one per language, each about a real codebase. Same question word for word, same model, same grader, same price list. squeezy answers from its local knowledge database of the repository. On the board where it saved less, the middle language cost 60 cents for every dollar the baseline spent.

0.60 the middle of 15 languages, on the board that saved less · the other board saved more

both boards

mini board 15 / 15

squeezy vs external baseline (Mini)

Both sides on gpt-5.4-mini, median of 10 runs. Every language came in cheaper at the baseline's recall or better.

haiku board 15 / 15

squeezy vs external baseline (Haiku)

Both sides on Claude Haiku 4.5, median of 10 runs. Every language came in cheaper at the baseline's recall or better.

Add up all 15 tasks on a board and squeezy's total is 43% less across the Mini board, 56% less across Haiku.

cost only counts when the answer passes · a cheap wrong answer cannot win

language ratios

Cost ratio by language.

Ratio is squeezy's cost divided by the baseline's, so 0.60 means squeezy spent 60 cents where the baseline spent a dollar. A language counts as a win only if squeezy found at least as much as the baseline did and came in at least 5% cheaper, so a cheap wrong answer can't win. These 15 languages are the ones this suite scores. squeezy indexes more than 40 languages and formats; the full list lives in the language docs.

mini board · gpt-5.4-mini on both sides

Dart 68% lower

0.32 $0.0161

JS 68% lower

0.32 $0.0080

Java 66% lower

0.34 $0.0373

TS 66% lower

0.34 $0.0135

Go 58% lower

0.42 $0.0146

Ruby 52% lower

0.48 $0.0100

C 43% lower

0.57 $0.0207

Python 40% lower

0.60 $0.0079

C++ 30% lower

0.70 $0.0576

Swift 28% lower

0.72 $0.0123

PHP 20% lower

0.80 $0.0317

C# 18% lower

0.82 $0.0291

Kotlin 18% lower

0.82 $0.0353

Rust 15% lower · recall 93.3%

0.85 $0.0246

Scala 14% lower

0.86 $0.0355

haiku board · Claude Haiku 4.5 on both sides

Go 86% lower

0.14 $0.0175

Python 72% lower

0.28 $0.0092

Scala 71% lower

0.29 $0.0803

Dart 65% lower

0.35 $0.0533

C# 62% lower

0.38 $0.0548

Rust 60% lower · recall 90%

0.40 $0.0576

JS 58% lower

0.42 $0.0202

Kotlin 57% lower

0.43 $0.0657

Ruby 57% lower

0.43 $0.0330

C++ 55% lower

0.45 $0.1002

C 49% lower

0.51 $0.0549

TS 44% lower

0.56 $0.1051

Java 39% lower

0.61 $0.1382

PHP 26% lower

0.74 $0.0961

Swift 22% lower

0.78 $0.0228

Both agents read the same repository at the same commit, answered the same question, and were scored by the same grader. On the Mini board both sides are also pinned to the same reasoning effort -- how hard the model is allowed to think before it answers -- because that setting moves cost on its own. The baseline's cost is recomputed from its own token counts against the price list squeezy prices itself with, so the ratio compares two agents rather than two vendors' invoices. squeezy's figure includes whatever it handed to a subagent.

15 languages · one question each, on a repository pinned to a commit

how to read it

same task

The question and the repo are matched

Each row is one hard code-navigation question, asked of a real codebase pinned to a commit -- nginx, Terraform, Laravel and Akka among them. List every subtype of a type, trace a call graph, classify how each implementation is written. The grader scores how many of the required symbols the answer actually found.

median

Ten runs behind every figure

Agent runs vary in both cost and what they find, so each number is the median of 10 runs. A run that returned no cost at all, from a timeout or a failed launch, is dropped rather than scored as a free win.

scope

What the numbers cover

This is one task suite on two model tiers, recorded once. Your own bill depends on the models you pick, your provider's rates, and the shape of your work.

your own bill depends on your models, your provider, and your work