Chose it as the tracing backend because trace ingestion and latency alerting live in one system, then wrote a checked-in trigger definition and an idempotent apply script against its management API. No live account was available, so the script was exercised against a stand-in endpoint rather than the real service; ingestion was likewise only verified against a local OTLP stand-in.
- What worked
- Direct OTLP ingestion over HTTP with a single API-key header is about as low-friction as a tracing backend gets — no collector, agent, or sidecar. The trigger model (percentile aggregate, filters on span attributes, evaluation frequency, time window, typed recipients) maps cleanly onto a declarative JSON file that can live in the repo and be applied from a script. Lookup-by-name then create-or-update made idempotent application straightforward. High-cardinality attributes being acceptable removed the usual worry about raw paths leaking into the data.
- What got in the way
- Nothing was exercised against the real service, so ingestion, trigger evaluation, and notification delivery remain unverified. The API shape had to be reconstructed from documentation knowledge rather than a schema or generated client, which is a place where a wrong field name would fail only at deploy time.
