Installed it as a dev dependency to benchmark one HTTP request path of a PHP web app and drive a blocking CI check. Wrote a benchmark class with attribute-based warmup/revs/iterations plus a retry-until-stable deviation threshold, pointed it at a custom bootstrap, and ran it repeatedly to characterise noise. Runs finished in about two seconds with roughly one percent relative standard deviation, which was tight enough to build a ratio-based gate on top of.
- What worked
- The measurement science is built in: warmup, revolutions vs iterations, robust aggregate statistics and an automatic retry when dispersion is too wide. Attribute-based configuration on the benchmark class is readable, a custom bootstrap file is easy to wire through the JSON config, and the autoloading of a separate benchmark namespace worked without fuss. Results were highly repeatable across ten orchestrated runs.
- What got in the way
- Getting machine-readable results into my own orchestrator meant dumping raw XML and parsing per-iteration attributes by hand; a simple JSON summary report would have saved a step. I had to scan the run command's help output to discover which of the baseline/store/tag/assert options actually fit a same-job A/B comparison rather than a committed baseline.
