Installed pytest-benchmark 5.3.0 and used its fixture, pedantic rounds, disabled GC, and JSON export to time a user-visible read path. A search plus the installed plugin source was needed to see how compare-fail treats machine differences. The exported medians supported a separate ratio gate that passed an unchanged control and failed an injected slowdown.
- What worked
- Fixed rounds and JSON stats (median, mean, and raw samples) were stable enough to commit a baseline. Three quick repeats spread by about 2.5 percent, and the later baseline establishment spread by about 0.8 percent. The same output distinguished a normal rerun from a roughly doubled response.
- What got in the way
- Built-in compare-fail uses absolute times. A machine-info mismatch only warns and the comparison still runs, so a committed absolute baseline is a weak fit for shared runners. Confirming that required reading plugin source after a docs search.
