Numbers and limits

Every figure published on this site, the word that says how it was obtained, and the four rules the code follows when it counts. This is the one page that carries the limits for all of the local tools, so other pages link here instead of repeating them.

Where a number comes from#

Each local tool labels its own output. Run the engine's committed fixture set and the label is in the report:

caveman-engine evals run

{
  "passed": true,
  "tokens_before": 10577,
  "tokens_after": 2416,
  "token_ratio": 0.7715798430556868,
  "probes": { "total": 12, "retained": 11, "recoverable": 1, "lost": 0 }
}

{
  "name": "json-tool-output",
  "content_type": "json",
  "tokens_before": 1366,
  "tokens_after": 293,
  "ratio": 0.7855051244509517,
  "basis": "inferred",
  "passed": true,
  "byte_recoverable": true
}

Captured from a caveman-engine build dated 2026-08-24; caveman setup --install today pins bin-v1.1.7. Trimmed to the fields this page is about. Counting is done offline with the o200k_base BPE, which is OpenAI's tokenizer and an approximation of any other vendor's. Ratios between two arms of the same run are meaningful; the absolute counts are estimates.

Vocabulary#

Three words, fixed meanings, never interchangeable. A summariser that blurs them makes every claim downstream wrong.

inferred#

A local, per-run estimate computed on your own machine from an offline token counter. Every public Caveman tool emits this word. It measures size, and it stays inside the run that produced it.

measured#

Traffic that was actually observed, rather than estimated from bytes. Observing a request tells you what was sent, and says nothing about the request you would otherwise have sent, so measured stops one step short of a saving.

verified#

A saving confirmed against a real bill by a hosted rollout system that ran both arms of the comparison. Caveman Cloud is the only source of it; every tool documented on this site emits inferred instead.

recoverable#

A lossy result whose original bytes were written to a content-addressed handle before the result was emitted, so retrieve(handle) returns the original byte for byte. Order is the point: the store happens first, and the smaller view is emitted only once that write succeeded.

Published numbers#

The regressed case in the wrap benchmark is an HTML page for which no compression transform applied, so the wrapping overhead was paid and nothing came back. It stays in the table because a table that drops its losses cannot be checked.

The content-type row is a target, not your result. Measure your own payload instead:

caveman-engine compress < yourfile > /dev/null

The smaller view goes to stdout and is discarded here; the report goes to stderr, and its ratio is what that file compressed by.

Latency#

Compression costs time on the machine doing it. Both figures below were measured on an Apple M3 Pro with binaries dated 2026-08-24, against the example files this site serves.

Each file went through caveman-engine compress ten times with a scratch CAVEMAN_CCR_DB. The column is the median wall clock of those ten runs, process start included. An empty payload takes 18.6 ms at the median on the same machine, so that is the floor in every row and the compression itself is the remainder.

The largest file, 62 KB of JSON, spends about 11 ms on compression. Run to run spread is wide at this scale: the same file varied by 6 ms across its ten runs, and engine/SKILL.md by 44 ms.

The proxy hop was measured separately, with caveman-proxy in record mode on loopback in front of a local stub that answers instantly, so nothing in the number is provider time. Fifty identical 223 byte Anthropic-shaped requests went through the proxy and fifty straight to the stub, each on one keep-alive connection. The stub answered in 0.12 ms at the median and the proxied request in 0.31 ms. Three rounds put the added median at 0.12, 0.19 and 0.28 ms.

Record mode forwards bytes unchanged, so that figure is the hop alone. A request the proxy compresses also pays the engine time above, on a payload the size of the request rather than the whole session.

The four rules#

Per run, never per month. A local tool counts one payload, one session you ran, or the history window you pointed it at. The furthest it projects is a per-day rate inside that window, because anything longer would depend on traffic nobody measured.

Tokens first, money only where the provider counted it. Prices depend on the model, the region, the cache state at the moment of the call, and your contract. Compression figures are token counts and stop there. The provider catalog holds the price table, and the hosted plane is where a price is applied to observed traffic.

Ranges beside averages. A median with no spread hides whether the result is solid. The wrap benchmark publishes its 95 percent interval next to its headline for that reason.

Negative results stay in. The regression above is published at the same size as the wins. So is the finding in the Adobe paper that compressing the human's prompt makes models answer longer and worse, which is why nothing here rewrites your prompts.

What caveman will not do#

What the local tools actually do, and what each behaviour means for a number you read.

Count it yourself#

caveman-engine compress < your-payload.txt   # one payload, before and after, with the basis
caveman stats                                # the local store, aggregated
caveman trial -- claude                      # one session with and without, then trial report

caveman trial needs its own proxy. Add the local tools has the two commands that go around it when persistent routing is already on.

Two fields in caveman stats need their definitions said once. basis is inferred for every row a local proxy wrote. requests_eligible_for_compression counts the recorded requests that reached the compression path as candidates: the proxy was in compress mode, a way back to the elided bytes existed, and the cache epoch allowed the rewrite. It counts candidates, not wins, so a request that was eligible and came out no smaller is still in it. The gap between it and requests is the traffic the proxy never tried to shrink.

What the CLI reports back to Caveman about any of this, and how to turn that off, is on Telemetry.