Browse documentation

Technical documentation / Reference

Profiling benchmarks

Only ignored benchmark/profiling workloads belong in this repository. Keep workloads next to their owning code and retain only support they use. No unit, integration, doctest or correctness-test CI suites.

Profiling hooks live in profiling/profiling_support child modules. Any #[cfg(test)] production struct field or hot-path block must name its consuming workload in a comment. Prefer deriving measurements from existing state; retained instrumentation means profiling builds can differ from release builds.

Run one workload

From repository root, copy an exact name from inventory. Confirm one workload ran rather than zero:

cargo test --locked --release --lib FULL_BENCHMARK_NAME -- --exact --ignored --nocapture --test-threads=1

List workloads without running them:

cargo test --locked --lib -- --ignored --list

Local batch

Review inherited variables before execution. FFFIND_BENCH_* and FFF_CONTENT_BENCH_* can select another repository, print paths/patterns/results or increase work. PROFILE_* and ISSUE_379_BENCH_REPETITIONS change workloads. Use default settings in a clean shell; never select private data for shared logs.

PROFILE_TUI_RENDER_BACKEND=test cargo test --locked --release --lib -- \
  --ignored --nocapture --test-threads=1

Batch selects 41 local workloads. Execution can take several minutes and uses temporary filesystem fixtures, repository reads and child processes. No benchmark CI workflow. Recheck inventory after changing workloads.

For isolated measurements, see CPU profiling, memory profiling, repeated setup and session replay reuse. Render aggregate overlaps individual scenarios intentionally; use --exact for isolated runs.

Inventory

Full benchmark nameMeasurement
context::profiling::cache::optimization_harness_measures_large_replay_and_cache_materialReplay 1,500 tool pairs and build cache material; checks item count and final call.
context::profiling::replay_cache::replay_snapshot_benchmark_growing_sessionsCompare growing-session replay scans/cache hits; requires at least 25% median speedup. Timing-sensitive; run on an otherwise idle machine.
providers::openai::profiling::optimization_harness_measures_large_chat_body_constructionBuild a 2,000-tool-pair request; checks message count without sending it.
rendering::highlight::profiling::profile_source_highlightingMeasure cold/warm Rust and Markdown highlighting; checks lossless plain-text projection.
sessions::write::profiling::session_snapshot_size_measurementMeasure snapshots of 64 KiB, 32 MiB and 128 MiB temporary files.
sessions::write::profiling::session_event_append_measurementAppend 500 single-event streamed-output batches in a temporary session root; report total/per-append elapsed time and observed counter writes. Counter observations run outside timed appends.
profiling::profile_harness::content_search::content_search_benchmark_compares_ffgrep_native_and_candidate_engineCompare search engines; also checks synthetic match, limit, context and case-sensitivity scenarios before timing.
profiling::profile_harness::find::find_benchmark_measures_cold_warm_filtered_and_baselineMeasure cold/warm path searches, filters, file/directory kinds and baseline traversal.
profiling::profile_harness::mixed::profile_mixed_massive_activity_renderMeasure 30,000 synthetic activity nodes; in-memory construction, not disk-history loading.
profiling::profile_harness::cpu::profile_cpu_discover_skillsMeasure synthetic skill discovery and first body read; checks discovery counts, diagnostics and successful read.
profiling::profile_harness::cpu::profile_cpu_rendering_heavy_transcriptProfile rendering from local transcript fixtures.
profiling::profile_harness::cpu::profile_cpu_sse_parserProfile local Responses and Chat Completions SSE parsing.
profiling::profile_harness::cpu::profile_cpu_startup_discoveryProfile local settings, instructions, skills, sessions and cached catalog discovery.
profiling::profile_harness::cpu::profile_cpu_tool_timeout_cleanupProfile child-process cleanup; launches a short-lived process, not a provider request.
profiling::profile_harness::cpu::profile_cpu_tui_streaming_simulationProfile synthetic assistant/activity updates and caches.
profiling::profile_harness::render::profile_tui_render_pipelineRun aggregate render scenarios; retain alongside individual runners for whole-pipeline comparison.
profiling::profile_harness::render::profile_tui_render_pipeline_heavy_transcriptIsolate heavy transcript rendering.
profiling::profile_harness::render::profile_tui_render_pipeline_idleIsolate idle rendering.
profiling::profile_harness::render::profile_tui_render_pipeline_modalIsolate modal rendering.
profiling::profile_harness::render::profile_tui_render_pipeline_resizeIsolate rendering across resized viewports.
profiling::profile_harness::render::profile_tui_render_pipeline_scrollIsolate transcript scrolling in the render pipeline.
profiling::profile_harness::render::profile_tui_render_pipeline_streamingIsolate rendering during synthetic streaming.
profiling::profile_harness::scroll::profile_scroll_popup_idleMeasure scrolling an idle subagent timeline.
profiling::profile_harness::scroll::profile_scroll_popup_selected_streamingMeasure timeline scrolling while selected child streams.
profiling::profile_harness::scroll::profile_scroll_popup_unrelated_streamingMeasure timeline scrolling while another child streams.
profiling::profile_harness::scroll::profile_scroll_transcript_deep_historyMeasure scrolling through long transcript history.
profiling::profile_harness::scroll::profile_scroll_transcript_long_replyMeasure scrolling one long assistant reply.
profiling::profile_harness::scroll::profile_scroll_transcript_selectionMeasure deep-history scrolling with active text selection.
profiling::repeated_setup_bench::profile_issue_379_repeated_setup_pathsCompare startup, profile/config discovery, provider selection and request-schema reuse; provider stub never sends requests.
tools::fs_cache::profiling::filesystem_cache_large_directory_measurementMeasure cached listing of 10,000 temporary files.
tui::controller::app::profiling::prompt_input::profile_typing_under_streaming_loadCheck draft edits reach rendered prompt while output remains queued; measure idle, streaming and 120-turn history cases. Scripts controller components, not complete event loop or physical display.
tui::controller::pacing::profiling::scroll_coalescing_burst_measurementMeasure bursts of 8, 128 and 1,023 wheel events; checks bounded coalescing and event accounting.
tui::render::activity::profiling::optimization_harness_measures_large_activity_detail_renderRender 5,000 output lines; checks retained body length.
tui::render::profiling::text_modal_viewport::text_modal_large_document_workloadCompare visible-only and full-document rendering at multiple scroll offsets.
tui::sessions::commands::profiling::session_store_load_measurementMeasure session listing, direct title lookup and switch-bounded read/hydration of five largest sessions in disposable PROFILE_SESSION_STORE copy; skips when unset. May rebuild sidecars in that copy; prints only sizes, counts and timings.
tui::render::transcript::profiling::transcript_test_backend_measurementMeasure 100 warmed frames over a 500-entry transcript.
tui::transcript_cards::profiling::cache::transcript_projection_and_thinking_measurementMeasure cached frames, focus/width changes, selection and thinking deltas.
tui::transcript_cards::visual_lines::profiling::profile_large_response_first_display_and_resizeMeasure fresh/cached/retained-width projection for large ASCII and Unicode responses; check visible body and text preservation after resizing.
tui::transcript_cards::visual_lines::profiling::streaming_viewport_large_response_workloadCompare visible-only projection, forced rebuild and full-format reference across small/large ASCII and Unicode responses.
tui::transcript_cards::visual_lines::profiling::profile_scroll_completed_replyMeasure cached scrolling through a completed long reply.
tui::transcript_cards::visual_lines::profiling::profile_scroll_history_updatesMeasure history scrolling while the transcript tail changes.

Limits

Assertions inside workloads validate measurement inputs; passing measurements do not establish runtime correctness or physical-terminal compatibility. Replay speedup threshold depends on timing; run on an otherwise idle machine. Synthetic profiles cannot qualify private-session loading, authenticated Claude CLI behavior or current TypeSafe service responses.

Prompts exceeding 65,535 logical rows still have a known cursor/hit-testing defect in upstream tui-textarea-2 viewport representation. Benchmark-only policy does not fix that defect.

Edit this page on GitHub ↗