Technical documentation / Reference

Prompt-cache audit scripts

Feature docs index · Repository README

Purpose

Maintainer scripts under scripts/ check prompt-cache readiness without changing provider runtime code. The static audit is offline by default. The live checker requires explicit opt-in for networked cold/warm experiments.

Static audit: scripts/audit_prompt_cache.py

Use this script before and after prompt-cache provider work:

python3 scripts/audit_prompt_cache.py --repo .
python3 scripts/audit_prompt_cache.py --repo . --json > /tmp/magi-code-prompt-cache-audit.json
python3 scripts/audit_prompt_cache.py --repo . --strict
python3 scripts/audit_prompt_cache.py --self-test

Checks cover:

• Stable prompt prefix ordering in agent conversation assembly.
• Obvious ephemeral system-prompt inputs such as timestamps, random ids, session ids, auth/account ids, or cwd-specific runtime state.
• Provider request cache-scope plumbing.
• Custom Responses prompt_cache_key support.
• OpenAI Codex cache-scope evidence.
• Chat Completions streaming usage telemetry request.
• Cached-token and cache-write usage parsing.
• Local ContextCache separation from provider prompt cache (N/A when no automatic recorder is found in agent source).
• Custom provider classification via use_responses_endpoint.

Status meanings:

Exit codes:

--json emits one object with schema_version, repo, generated_at, overall_status, strict, and findings. Each finding includes stable fields such as id, status, severity, category, provider, file, symbol, evidence, expected, impact, suggested_fix, and references.

Use the audit output for findings on the checked revision rather than a fixed result count. context_cache.separation is N/A when no automatic disk snapshot recorder is found; retained local cache readers are not provider prompt-cache evidence. A codex.cache_scope failure means the audit did not find its recognized cache-scope field, not that live Codex caching fails.

Implemented static evidence includes:

• request.cache_scope: ProviderRequest carries a crate-private stable prompt cache key.
• responses.prompt_cache_key: custom Responses bodies emit prompt_cache_key when request scope exists.
• codex.cache_scope: the audit does not find its recognized cache-scope field in the Codex request body or headers; backend acceptance and scope remain unverified.
• usage.cached_tokens.chat: Chat Completions prompt_tokens_details.cached_tokens maps to Usage.cache_read.
• chat.stream_usage: streaming Chat Completions requests include stream_options.include_usage so providers can return usage telemetry.
• Agent session runs derive bounded cache scope as magi-code-session-{first_32_hex_sha256(session.id())}. Raw session ids, cwd, account data, timestamps, random values, and prompt text are not included.

Chat Completions bodies intentionally do not emit prompt_cache_key; only Responses-compatible request shapes receive that field.

Do not interpret PASS findings as proof of live provider cache hits. The static audit proves source evidence only; Codex cache scope remains unresolved.

Live checker: scripts/check_prompt_cache_live.py

Use this script only when you intentionally want a networked provider check:

python3 scripts/check_prompt_cache_live.py --help
python3 scripts/check_prompt_cache_live.py \
  --url https://example.invalid \
  --shape responses \
  --body-file scripts/fixtures/prompt_cache_audit/live_offline_body.json
python3 scripts/check_prompt_cache_live.py \
  --allow-network \
  --url "$URL" \
  --shape responses \
  --body-file /tmp/prompt-cache-live-body.json \
  --bearer-env OPENAI_API_KEY \
  --repeat 2 \
  --expect-warm-cache

Safety behavior:

• Network is denied unless --allow-network is present.
• Without --allow-network, send attempts exit 2 before reading secrets.
• Secrets are read only from named environment variables such as --bearer-env and --account-id-env.
• Script never reads ~/.magi-code/auth.json or provider settings automatically.
• Secret-bearing --header names such as authorization, api-key, cookies, and chatgpt-account-id are rejected; use env flags instead.
• Output redacts bearer tokens, API keys, cookies, tokens, and account ids.
• Request body is read once after the network gate, kept as bytes, and reused for each repeat. Output reports body_sha256 and body_bytes.
• JSON and SSE data: responses are parsed for cached-token fields. Raw request headers are not logged.

Supported shapes:

Exit codes:

Verification

Credential-free checks:

python3 scripts/audit_prompt_cache.py --help
python3 scripts/check_prompt_cache_live.py --help
python3 scripts/audit_prompt_cache.py --self-test
python3 scripts/check_prompt_cache_live.py --self-test
python3 scripts/audit_prompt_cache.py --repo .
python3 scripts/audit_prompt_cache.py --repo . --strict
python3 scripts/audit_prompt_cache.py --repo . --json

Any FAIL finding makes both normal and strict audits exit 1. With no FAIL findings, WARN exits 0 normally and 1 under --strict. Static findings do not establish live provider cache behavior; missing recognized cache-scope evidence is not proof that live Codex caching fails.

Back to feature docs · Back to repository README

Edit this page on GitHub ↗