Used its cachegrind tool to count retired instructions for an optimized binary across four command shapes, so a CI gate could compare against a committed baseline instead of wall-clock timings. Repeat runs were bit-identical, versus roughly five percent variance on wall-clock, and an injected hot-path allocation of about four percent tripped the threshold cleanly.
- What worked
- Determinism is the whole reason this approach works on noisy shared runners: identical counts to the hundredth of a percent across runs, yet sensitive enough to catch a single extra allocation per string value. Simple command-line invocation per process, easy to parse the summary count out of.
- What got in the way
- Counts are not portable across versions, so a baseline recorded with one release is meaningless against another. That forced me to record the environment fingerprint into the baseline file and add a mismatch guard, and it means the real baseline has to be seeded on the CI image rather than locally. Simulation overhead also makes each measured run far slower than running the binary natively.