Memory Usage Profiling Workflow

Maintainer workflow for credential-free peak RSS checks. Default path is offline: scripts/profile_memory.py runs the same ignored Rust profiling harness tests as CPU profiling with a temporary synthetic MC_HOME, no auth.json, credential-looking environment variables removed from child processes, local fixtures, and no live provider/network command.

Requirements

Rust/Cargo and python3 on a Unix-like host with Python resource module.
No third-party Python packages.
Optional deeper tools are not part of default workflow.

Default credential-free run

python3 scripts/profile_memory.py --repo . --all --iterations 1

Artifacts:

target/profiling/memory/profile-results.json
target/profiling/memory/profile-summary.md
target/profiling/memory/<scenario>.log

Run one scenario and print machine-readable JSON:

python3 scripts/profile_memory.py --repo . --scenario provider_sse_parser --iterations 1 --json

Self-test:

python3 scripts/profile_memory.py --self-test

Scenarios

Memory scenarios reuse CPU harness workloads; only the measurement wrapper changes. The Rust test names stay profile_cpu_* because they are shared synthetic workloads.

scenario	harness	exercises
startup_discovery	`profile_cpu_startup_discovery`	local settings, instruction, skill, session, and model-cache discovery
provider_sse_parser	`profile_cpu_sse_parser`	Responses and Chat Completions SSE parser fixtures
rendering_heavy_transcript	`profile_cpu_rendering_heavy_transcript`	transcript rendering, markdown, code highlighting, diff/plain projection
tui_streaming_simulation	`profile_cpu_tui_streaming_simulation`	assistant deltas, activity events, transcript/activity caches
tool_timeout_cleanup	`profile_cpu_tool_timeout_cleanup`	Unix process-group timeout cleanup path

Metrics

profile_memory.py warms the release test build first with cargo test --release --no-run, then runs each selected scenario in an isolated helper process. The helper reads resource.getrusage(RUSAGE_CHILDREN).ru_maxrss after the scenario command exits.

JSON fields:

max_rss_bytes: peak resident set size normalized to bytes.
max_rss_mb: max_rss_bytes in MiB, rounded for reading.
elapsed_ms: workload elapsed time reported by the shared Rust harness.
command_elapsed_ms: wall time for the measured cargo test --release <scenario> command.
primary_metric / metric_value: workload-specific CPU harness metadata kept for comparison context.

Unit normalization: Darwin reports ru_maxrss in bytes; Linux reports KiB. The script normalizes both to bytes.

Comparison method

Compare only runs with the same machine, OS, Rust version/profile, and allocator, under similar system load.
Run:

python3 scripts/profile_memory.py --repo . --all --iterations 1

Save measured rows from target/profiling/memory/profile-results.json with the run timestamp, commit, platform, Rust version and profile in your comparison record. Preserve earlier measurements; mark missing values unknown rather than inferring them. Keep raw artifacts out of Git.
Investigate a >5% peak RSS increase versus the previous comparable run; it is not a CI gate.
Use profile-summary.md for quick review and scenario logs for harness output.

Memory numbers are not comparable across OSes, allocators, Rust versions, debug/release profiles, or machines.

Optional deeper inspection

The default workflow above is credential-free and offline. Live-provider memory checks are outside that workflow and involve secrets: run them only when credentials and network are intentionally available, and never store secret strings, account identifiers, auth flow codes, or credential payloads in fixtures or artifacts.

Optional local tools when installed:

# macOS
/usr/bin/time -l cargo test --release profile_cpu_rendering_heavy_transcript -- --ignored --nocapture
leaks <pid>
# Instruments Allocations can inspect a manually launched process.

# Linux
/usr/bin/time -v cargo test --release profile_cpu_rendering_heavy_transcript -- --ignored --nocapture
heaptrack cargo test --release profile_cpu_rendering_heavy_transcript -- --ignored --nocapture
valgrind --tool=massif cargo test --release profile_cpu_rendering_heavy_transcript -- --ignored --nocapture

Optional tool output belongs under target/profiling/memory/ or another ignored local path.

Edit this page on GitHub ↗