Offline tool-output measurement
Feature docs index · Repository README
Purpose
Estimate potential tool-result token savings from stored sessions without changing live provider requests.
Run
Run the exploratory measurement prototype explicitly with:
magi-code sessions measure-compression --model <MODEL>It reads only active primary JSONL files directly under sessions/ and active subagent JSONL files directly under sessions/subagents/. Discovery traverses the absolute session path component by component from a stable filesystem anchor without following ancestor or root symlinks, validates each held directory handle, opens subagents/ without following a symlink, enumerates both directories through held directory capabilities, and opens each candidate read-only relative to the matching directory capability. Selected files are not reopened through ambient path access. Only active-name regular files with stable identity metadata and no more than the internal 64 MiB per-file cap become eligible. Normal archives, sidecars, temporary files, history directories, and other non-candidates are ignored without skipped-input counts. Symlinked active JSONL candidates, unsafe active file types, and an unsafe subagents/ path are rejected and counted as skipped. The command emits one aggregate JSON report with counts and local token estimates; it does not emit session paths, IDs, raw output, excerpts, or file contents. The report is privacy-oriented and read-only.
Only supported openai-codex text model families are accepted; embedding models are not. The o200k_base family accepts exact gpt-5, gpt-4.1, gpt-4o, gpt-4.5, o1, o3, and o4 families with either the exact family name or a valid - suffix; gpt-5 also accepts version suffixes that begin with .. codex- models require a nonempty valid identifier after the prefix. The cl100k_base family accepts exact gpt-4 and gpt-3.5-turbo families with the same valid - suffix rule. Known non-text modality identifiers (embedding, audio, realtime, transcribe, tts, and image) are rejected. These are local estimates, not provider billing counts. An unsupported model is rejected without scanning. Optional --max-files and --max-bytes flags bound sampling. --max-files defaults to 100 and has a maximum of 10,000. --max-bytes is a byte count; it defaults to 128 MiB and has a maximum of 1 GiB (with a 1 MiB minimum).
This prototype does not change provider-visible output, sessions, settings, authentication, or files under MC_HOME. It does not use the network, enable any production codec, or implement TOON.
Measurement report
The JSON report is aggregate-only. Its fields are:
| Field | Meaning |
|---|---|
prototype | Always offline_compression_measurement_v0; identifies this exploratory report, not a production codec or setting. |
schema_version, model, encoding_family, scope, exploratory | Report version, requested model, selected local BPE family, fixed active-session scope, and the fixed exploratory marker. |
measurement_unit | Always unique_stored_tool_results; measurements are reported per stored tool result rather than per provider request. |
request_weighted | Always false; results are not weighted by request volume. |
replay_compaction_applied | Always false; no replay compaction is applied before measurement. |
sampling | File and byte sampling totals. Eligible files are opened and measured from held read-only handles; rejected unsafe files and files over the internal 64 MiB per-file cap are not part of these totals. |
files.considered | Eligible active regular JSONL candidates after safe opening and the per-file cap, before sampling. scanned is the number selected and whose handles passed the scan checks; sampled_out covers eligible files omitted by the file/byte limits. skipped covers rejected active candidates such as symlinks; unreadable, unstable, and limit_hit are separate safety/limit counts. limit_hit includes file-level input, result, candidate, and per-file bounds. |
jsonl.lines, valid_events, malformed, oversized, limit_hit | Bounded JSONL scan counts. oversized counts lines over the line cap; limit_hit counts files that reached the line-count cap. |
tool_results.found, empty, measured, legacy, invalid, skipped_limit | Tool-result counts. Empty results are found and measured when their complete item framing is available, but normally have no codec candidate. invalid counts found tool-result events that fail required-field or framing validation; skipped_limit counts only known tool results over the candidate-size cap. |
baseline | Raw content bytes and Codex Responses-style local BPE projection tokens for each function_call_output item before a codec; these are not full provider-context or billing-token counts. |
codecs | Per-codec attempts, validated candidates, strict improvements, selections, candidate token totals, and selected savings. |
portfolio | Tokens selected after independent per-result selection, saved tokens, transformed output count, and savings_percent. The percentage is rounded to two decimal places and is 0.0 when the baseline is zero. |
warnings | Fixed, non-sensitive scope, quality, safety, and limit warnings. |
The sampling byte fields use opened-handle snapshot lengths, not physical I/O counts. The accounting equations are:
discovered_files = selected_files + sampled_out_files
discovered_bytes = planned_bytes + sampled_out_bytes
planned_bytes = scanned_bytes + unscanned_selected_bytesdiscovered_bytes is the sum for eligible files before sampling; planned_bytes is the sum for whole files selected by the file and byte limits; and sampled_out_bytes is the eligible sum omitted by those limits. A selected file contributes its snapshot length to scanned_bytes only when its handle passes identity/length checks before and after scanning. An unreadable or unstable selected file contributes that same expected snapshot length to unscanned_selected_bytes, even if a bounded scan read some bytes before failing. A stable scan can still set files.limit_hit when the JSONL line cap stops parsing; its full snapshot length remains part of the planned/scanned accounting.
The codecs are independent: JSON minification removes only lexical JSON whitespace and preserves JSON values but not original bytes; line RLE repeats only adjacent identical complete line chunks; and template folding stores one conservative prefix/suffix span for an adjacent run. RLE and template candidates are decoded locally and must reconstruct the exact input. JSON candidates are validated as complete JSON, not against the original bytes. Only a unique candidate with fewer full-item BPE tokens than baseline is selected. Candidates never stack, and ties fall back to baseline.
JSON minification has a lexical/value-preserving guarantee, not an original-byte round-trip guarantee: it removes only JSON whitespace outside strings and keeps string escapes, duplicate keys, key order, and numeric spelling. The baseline and every candidate are counted as a Codex Responses-style function_call_output item containing type, call_id, and output; stored tool names and success flags validate the source but do not change that count.
The RLE and template envelopes include fixed instruction strings. Those strings are part of the candidate output and therefore part of its token count. Their decoders require the exact marker and instruction, canonical JSON, known fields, bounded item expansion, and exact reconstruction before a candidate is eligible. The template codec is a conservative local heuristic, not evidence that a live provider will natively decode the envelope. Sampling is bounded and deterministic: newest eligible opened handles are retained within the file cap, files larger than --max-bytes are omitted before they consume a selected slot, and the remaining candidates are planned in newest order without exceeding the byte cap. No selected file is split. The resulting report is a sampled, unweighted experiment rather than a billing or request-volume estimate.
The source JSONL is never copied into the report or another file. It is opened read-only, scanned within line/file/result bounds, and discarded after counting.