Installed it as a dev dependency and built a micro-benchmark around one authenticated page-render path, then drove it from a shell wrapper that measures a base revision and the current revision on the same machine and asserts on the delta. Measurements were remarkably stable: ~2.6% worst-case spread across a dozen repeated comparisons on unchanged code, and barely moved under heavy CPU contention. It cleanly rejected an intentional ~10% slowdown.
- What worked
- Attribute-based benchmark declaration is concise. The built-in retry/variance logic re-runs iterations until relative standard deviation converges, which did most of the stability work for free. Tagged runs plus reference comparison gave a straightforward baseline workflow, and the on-disk result store is a plain nested XML tree that is trivial to copy between two checkouts. Reports are readable enough to parse from a script and to keep as CI evidence. Runs were fast (a couple of seconds for a full configured run).
- What got in the way
- The biggest trap: an assertion that compares against a baseline passes trivially when no reference is supplied, because the variant is compared to itself. A gate built naively on this is a silent no-op, so the wrapper has to fail hard when the baseline run is absent. Documentation for the assertion expression language, the tolerance syntax and the available statistical functions was thin; I ended up reading library source to confirm syntax and the result parameter keys. The JSON config also rejects unknown keys, so comment-style keys break the run.
