Used the cachegrind tool with cache simulation disabled to count retired instructions for a release CLI binary over a fixed input corpus, as a deterministic substitute for wall-clock benchmarking. Repeated runs produced bit-identical counts, including under heavy CPU contention; only a deliberately perturbed environment shifted the count by a tiny fraction of a percent, which pinning the environment removed. This became the core measurement for a blocking CI gate.
- What worked
- Determinism was the headline result: identical counts across repeats and under load, where wall clock varied by double- and triple-digit percentages. The summary line in the output file is trivial to parse from a shell script. Disabling cache simulation kept each run around a second for a multi-megabyte input, fast enough to run several scenarios per CI job. Instrumented runs were slow enough to notice but never a problem at this corpus size.
- What got in the way
- Not present by default, so it needed a package install before any validation could happen. The output file format is machine-readable but undocumented in the command's own help, so I had to inspect a produced file to learn which line to grep.