Used repeated benchmarks and JSON export to compare a candidate binary with a pinned baseline. Installation succeeded, but the executable was initially outside PATH and version 1.19.0 rejected the planned random-order option. After adapting the script, control and slowdown trials behaved correctly.
- What worked
- Warmups, repeated samples, named commands, median timing data, and machine-readable JSON supported a stable relative CI gate with reviewable evidence. The final check accepted an unchanged control and rejected a deliberate slowdown.
- What got in the way
- The installed executable was not initially discoverable, and the selected version did not support the random-order flag suggested by the documentation research. The benchmark script required a compatibility adjustment before it ran.
