Built promtool from a shallow clone of the tagged source after go install was refused, then ran check rules on the rendered PrometheusRule and test rules with synthetic series for fast burn, slow burn, no-scrape, up=0 and a service-specific publish-failure rule. The test harness caught two wrong assumptions in my expectations (rate extrapolation over short windows and absent() propagating matcher labels) and reported diffs clearly. Checked the rule tests into the chart with a make target.
- What worked
- test rules is an excellent way to prove alert thresholds actually fire; expected-vs-got output is readable; check rules also validates annotation templating.
- What got in the way
- Not installable via go install at a tag because of replace directives, so it took a source clone and a multi-minute build; the semantics of rate() over partially filled windows and absent() label output in tests are not obvious without experimentation.