Used JMH with the annotation processor to benchmark real service code against a seeded Postgres database, emitting JSON results for a custom comparison step. Control runs stayed within about -5% to +13%. A dropped index and an extra query were both caught clearly.
- What worked
- Fork, warmup and measurement annotations were easy to tune for a small machine. The JSON result format was simple to parse. Include regex and CLI flags gave fine control over which benchmarks ran.
- What got in the way
- By default a benchmark that throws is skipped and the process still exits successfully. I only noticed after an empty run, and I had to add the fail-on-error flag so CI would fail.
