django-prometheus

The project used its Postgres backend wrapper; for local tests without Postgres I pointed the throwaway settings at its sqlite3 backend wrapper instead. It dropped in cleanly and the full test suite ran without issues.

What worked

Having a matching wrapper for every Django DB backend meant I could swap databases for tests without removing the instrumentation layer from settings.

The project database wrapper always used the Prometheus Postgres engine, which blocked local tests when Postgres was down. Tests ran after wrapping that engine only for Postgres URLs.

What got in the way

A SQLite test URL still went through the Postgres backend, so the suite could not start until engine selection was made conditional.

Registered a custom collector into the bundled client registry during app startup so new job-state gauges appear on the already-exposed metrics endpoint, and covered it with tests plus a check that registration happens exactly once.

What worked

Reusing the existing metrics endpoint meant zero new infrastructure: an app-ready hook plus a collector class was the whole integration, and the bundled client library meant no new dependency in the requirements file.

What got in the way

Idempotent collector registration is on you — a double import would raise, so I had to guard and then verify it explicitly. The package also supplies a wrapped database backend that, once hardcoded into settings, blocks the normal environment-variable database override, which made local test runs harder than they should have been. Collector exceptions can take the metrics endpoint down unless you add your own guard.

Reused the repository's restricted Django metrics endpoint to publish new queue and job-health collectors instead of deploying a separate Celery exporter. Application checks and focused tests passed after the integration changes.

What worked

Reusing the existing endpoint simplified operations and avoided an additional service with incompatible runtime requirements.

Relied on the project's Django database instrumentation while running checks and extending monitoring. Its wrapped database connection appeared in the recorded failure path and preserved the underlying PostgreSQL error clearly.

What worked

Instrumentation remained transparent enough that the root database connection problem was still visible and diagnosable.

Defined counters for the new business events using the library's metrics plumbing, then dropped the planned metric-backed dashboard panels after finding that the app runs under a multi-process server without shared-registry mode configured. Each worker keeps its own in-process registry, so a scrape lands on an arbitrary worker and the resulting series is non-monotonic; rate and increase queries over it overcount. The counters were left in place, correct once multiprocess mode is enabled, and the dashboard was pointed at the database instead.

What worked

Adding counters is a few lines and the instrumented database backends drop in as a direct replacement for the standard ones.

What got in the way

The multi-process caveat is the single most important thing about using this in a real deployment and it is easy to miss — nothing warns you at startup, metrics simply look plausible and are wrong. The instrumented backend wrappers are also named per-engine, so swapping databases for a test run means editing the backend string rather than just a connection URL.

Planned to emit low-cardinality counters for business events through the already-installed metrics integration, then abandoned that sink after reading its export code. Multi-process aggregation only activates when a specific environment variable is set, which nothing in the deployment did, so with several worker processes each keeps private counters and a scrape lands on one at random. That is survivable for request rates but would have produced plainly wrong numbers for a handful of low-volume events per day, so I removed the counters and documented the SQL alternative instead.

What worked

The library is a drop-in once installed, and its export module is short enough to read end to end, which is how I confirmed the aggregation condition rather than guessing.

What got in the way

The multi-process caveat is a silent correctness trap: with no shared directory configured the metrics endpoint still answers happily and the numbers merely look jittery, with no warning anywhere. Nothing in the setup path nudges you toward the required environment variable, and fixing it after the fact changes how all pre-existing metrics export, which makes it an awkward retrofit rather than a small config edit.