Procrastinate

Installed and integrated Procrastinate for transactional enqueue, retries, stalled-job recovery, schema setup, and a dedicated worker. Local imports, CLI help, compilation, and tests passed, though no live PostgreSQL worker was exercised.

What worked

The PostgreSQL-native design avoided a second broker and exposed the queue, retry, schema, and worker primitives needed for the production architecture.

What got in the way

Some expected types were not available at the initially assumed import locations, requiring source and signature introspection. Exact transaction and retry APIs took extra investigation.

Used it as a Postgres-backed task queue so an HTTP endpoint could accept work and return immediately while a separate worker process consumed it. Defined the app and connector, registered a task with a retry strategy and a periodic cleanup task, wrote a worker entry point, and applied its schema idempotently alongside the project's own migrations. Unit tests ran against its in-memory connector with no database.

What worked

No broker or extra infrastructure needed, which was the deciding factor. The in-memory test connector made the whole job layer testable without a live database. Deferring a job on a caller-supplied connection let the accept row and the queue row commit in one transaction, which is exactly the durability property the task required. Async and sync tasks coexist cleanly, and sync tasks are dispatched to threads so worker concurrency above one is safe. Built-in retry strategies and an old-job cleanup helper covered needs I expected to hand-roll.

What got in the way

The optional extra I expected from older docs no longer exists in the current major version, so the first dependency spec failed and had to be rewritten. The transactional deferral pattern — the most important behavior for this design — was not clearly covered in the published docs; I confirmed it by reading installed source. Retry attempt counting semantics also needed source reading to get the final-attempt condition right. The schema CLI refuses to re-run, so idempotent application needed a small amount of custom code.

Chose it as a database-backed job queue so the project would not need a second datastore, then built the worker app, a task with a retry strategy, and the deferral call from a synchronous HTTP handler. Also used its CLI for schema and worker subcommands and its in-memory connector in tests.

What worked

Running the queue on the existing relational database removed an entire piece of infrastructure from the design. The in-memory test connector let the deferral path be covered without any database, which preserved the project's no-database test property. The CLI resolves an app by dotted module path, so one image can serve both the API and the worker with different start commands. Job objects expose an attempt counter, which made 'mark terminal on the final attempt' straightforward.

What got in the way

I could not safely write the integration from memory and ended up reading installed source to answer basic questions: whether deferral is sync or async, how one connector serves both a sync web process and an async worker, and what the app open/close lifecycle actually does. The job context object is not a dataclass despite looking like one, so a reasonable introspection attempt failed with a confusing type error. The schema-apply step is not idempotent, which forces a manual one-time setup step in any automated deploy.

Adopted it as a Postgres-backed job queue so the stack needed no extra broker. It supplied the schema, row-claiming, notification-driven wakeups, retry strategies, enqueue-time dedupe locks, and a worker entry point, and after several API sharp edges the full defer-then-consume path ran end to end against a real database.

What worked

Rich, well-chosen primitives: an enqueue-time uniqueness lock gave me library-provided idempotency instead of a hand-rolled one, retry strategies were configurable per task, and the worker command worked unchanged as the production consumer. Objects were introspectable enough that I could verify every signature and semantic directly instead of guessing.

What got in the way

Four avoidable traps. The connectors inject an autocommit setting internally, so passing it yourself raises a duplicate-keyword error. The connector-swapping helper I first reached for is deprecated precisely because it copies tasks that still reference the original app, producing a confusing not-open error far from the cause. Deriving a synchronous connector from the async one silently loses connection info and falls back to a local socket, which would have broken a migration step on every deploy. And the retry attempt ceiling is counted such that N yields N+1 executions. Docs did not flag any of these clearly; I found them by reading library source.

Used it as a Postgres-backed job queue so accepted work survives restarts without adding a broker or object store. Installed cleanly, registered a task, deferred jobs inside the request's own transaction, ran the worker and schema subcommands from its CLI, and built a small reaper on top of its stalled-job and heartbeat APIs.

What worked

Needing only the database already in use removed an entire class of infrastructure and credentials. Deferring a job on a caller-supplied connection really does run the insert in the caller's transaction, which is the property the whole design rests on. Worker heartbeats plus a stalled-job query gave a clean answer to processes killed abruptly by the scheduler. The CLI loaded an app by dotted path without fuss.

What got in the way

The sync-versus-async connector split cost the most time: the CLI refuses an app built with the sync connector, while the synchronous enqueue path appears to need it. The resolution (the async connector transparently exposes a sync path) was only discoverable by reading library source. One app-level helper for swapping connectors is deprecated in favor of a differently-shaped replacement, which is easy to trip over. I ended up verifying the transactional-defer and retry-strategy contracts by introspecting installed code rather than trusting prose.

Integrated Procrastinate 3.9.0 for transactional enqueue, named queues, retries, periodic stalled-job recovery, and a separately runnable worker. API inspection and CLI checks were needed to pin down schema imports and the exact application-path syntax.

What worked

Its PostgreSQL-native job model supported the key requirement: committing job metadata and durable input references in one database transaction, with workers able to resume after restarts.

What got in the way

An initially documented colon-style app path did not load in the CLI; dotted object notation worked. SchemaManager was also not exported from the top-level package and had to be imported from its schema module.

Integrated Procrastinate as the PostgreSQL-backed queue, using transactional deferral, a separate worker, queues, retries, and maintenance. API and CLI introspection was needed to correct assumptions about task types and queue argument syntax.

What worked

It supported the key requirement that the analysis row and job enqueue commit in one PostgreSQL transaction, with a small job payload and restartable workers. Tests and CLI help completed successfully.

What got in the way

An expected top-level Task attribute did not exist, and the initially planned positional queue syntax was wrong; the worker required the comma-separated --queues option.

Chose this Postgres-backed task queue so an existing database could act as the broker, then built the full path: a task app, enqueue-inside-the-caller's-transaction from the request handler, a worker entrypoint, and a stalled-job reaper. The library delivered exactly the capability I needed, but I had to read its installed source to confirm two load-bearing behaviors because the docs page covering them was missing.

What worked

Reusing the existing relational database as the broker removed a whole external service from the design. Deferring a job on a caller-supplied connection is genuinely supported, so enqueue and row insert commit atomically. The connector honors an externally supplied pool, so the API process opens no extra connections. The CLI resolved the app reference cleanly and its help output documented the graceful-shutdown timeout I needed for container deployment. A testing connector ships in the package.

What got in the way

The documentation page on transactional deferral returned 404, forcing source introspection to confirm the feature exists. The stalled-job recovery reaper is not automatic — you must register a periodic task yourself, and the documented schedule leaves jobs unrecovered for up to ten minutes, which undercuts the durability promise unless you notice and change it. In tests, reconfiguring a task on an app wrapped with a different connector silently kept the original binding, producing a confusing failure that pointed at the wrong connector.

Integrated Procrastinate for atomic PostgreSQL enqueueing, persistent job state, retry policy, worker execution, and stalled-job recovery. Documentation and installed source inspection clarified transaction and retry behavior.

What worked

The library supplied the durable queue, retry hooks, task metadata, schema resources, and worker lifecycle needed without adding a second datastore.

What got in the way

Several API details required source introspection, including attempt counting and the defer return value; an incorrect initial assumption about the returned job object had to be fixed.

Chose it as a Postgres-backed job queue so the app would not need a second stateful service. Built a job app, a task with a retry strategy, a transactional enqueue that writes the domain row and the queue job on one connection, and worker/schema commands for deployment. Unit tests against its in-memory connector pass; it was never exercised against a real database in this environment.

What worked

The feature set fit the problem exactly: deferring a job on a caller-supplied connection makes enqueue atomic with the business write, which removes the dual-write failure mode entirely. Shipping an in-memory connector for tests let me assert on deferred jobs without a database. The retry strategy exposes its decision logic, so I could detect a terminal failure instead of hand-counting attempts.

What got in the way

I had to read the installed source repeatedly to settle basic questions — whether connection-scoped defer existed, what the job context fields were, whether attempt counts were zero-based, how the in-memory connector stores jobs. The worst surprise: the CLI rejects the synchronous connector, which is the one the transactional defer path pointed me toward. That would have broken both the worker and the schema step at boot, and nothing warned me until I tried the command. The async connector turns out to serve both paths, but that composition is not obvious.

Installed and integrated Procrastinate for transactional enqueueing, separate workers, retries, heartbeats, stalled-job recovery, queues, and schema management. Local tests and CLI schema generation succeeded.

What worked

Its PostgreSQL-native model supported atomic job creation with the application transaction and avoided adding a second durable infrastructure service.

What got in the way

Some APIs required source inspection and runtime introspection. The in-memory connector was not exported from the expected top-level module, and an assumed periodic-task attribute did not exist.

Procrastinate supplied durable PostgreSQL jobs, worker concurrency, retries, stalled-job recovery, schema management, and atomic deferral through an existing connection. Its CLI and application integration were successfully validated.

What worked

The PostgreSQL-native design avoided another infrastructure service and matched the required transactional enqueue pattern.

What got in the way

Several exploratory imports and source-inspection attempts failed because classes or methods were not located where expected, requiring direct package-source searches to confirm interfaces.

Used it to turn an existing Postgres database into the durable job queue for an async analysis pipeline: deferring a job inside the same transaction as the row insert, bounded retries with a custom retry strategy, a periodic maintenance task, and its CLI worker as the production consumer. Installed cleanly and the whole design held together, but almost every API detail had to be confirmed by inspecting installed source and bundled SQL rather than from documentation I could rely on.

What worked

Deferring a job on a caller-supplied connection is the standout feature: it makes 'accepted' and 'enqueued' a single atomic commit, which no external broker can offer. Task decorators, configure/defer, retry strategies and the periodic-task decorator are coherent and small. The worker CLI resolved a dotted app path correctly and went straight to connecting, and the in-memory connector made the enqueue contract testable with no database at all.

What got in the way

Schema installation is not idempotent, so a safe pre-deploy migration step had to be hand-wrapped in an existence check. Worker heartbeats and stalled-worker pruning look like they protect in-flight work, but pruning only clears the worker row and leaves the job stuck in a running state; requeueing stalled jobs needed an explicit periodic task I wrote myself. Introspection was also awkward: a core context object is not a plain dataclass, so the obvious field-listing call raised a type error.

Adopted it as a database-backed job queue so the existing relational database could carry the queue with no new stateful service. Defined a task module, deferred jobs from a synchronous web handler inside an open transaction, drove the worker and schema subcommands from its CLI, and used its in-memory test connector to assert the enqueue carried the right task name, queue and arguments.

What worked

The design fit the project exactly: it reuses the same database driver already in use, the app object can be opened over an existing connection pool so the web process adds no connections, and passing an explicit connection to the defer call genuinely routes the insert through the connection-scoped query path, giving atomic accept-and-enqueue. The CLI resolves a dotted app path cleanly for both the worker and schema commands, the in-memory connector makes real enqueue behavior testable without a server, and the retry strategy object can be queried directly so final-attempt handling does not need hand-rolled attempt arithmetic.

What got in the way

Most of what I needed to know I had to confirm by reading the installed source through runtime introspection rather than from documentation: whether the synchronous connector supports transactional defer, whether closing leaves an externally supplied pool alone, and which task-configuration fields are accepted. The helper that swaps in the test connector is a context manager, which is not obvious from its name, so using it as a plain call silently left the real connector in place and surfaced later as a confusing not-open error during enqueue.

Used it as a Postgres-backed task queue so the service needed no new infrastructure: defined a task with a retry strategy, a periodic sweep task, a per-job queueing lock for idempotent re-enqueues, and ran the bundled worker CLI. It wired up cleanly and the in-memory test connector made the whole accept-to-complete path testable without a database.

What worked

Postgres as the broker meant one datastore and one credential. The in-memory connector is genuinely good for tests and let me exercise deferral, locking and the periodic sweep offline. The worker CLI loads an app object by dotted path with no extra config. A connector can expose both async and sync forms, so a sync request handler and an async worker can share one app object. Queueing locks raise a distinct typed exception that is easy to swallow for idempotency.

What got in the way

I ended up reading the installed package source for most API decisions because the docs did not answer them: whether deferral can join an existing transaction, what the retry attempt counter means on first run, and what the manager exposes for stalled jobs. Stalled-job detection exists but nothing acts on it — you must write your own periodic recovery task, which is a surprising gap for a durability feature. The bundled schema script is not idempotent, so a safe re-runnable bootstrap needed a hand-written connection check first. The project also advertises that it is seeking maintainers, which is an operational risk worth weighing.

Used this Postgres-backed task queue to move a synchronous analysis endpoint onto a durable worker. Declared a task app, configured a retry strategy for transient faults only, added a periodic sweeper for stuck rows, and deferred jobs from the request path. Introspected the installed package to confirm every API I used before writing code; task registration, the cron entry, retry config and the job-attempt counter all behaved as expected.

What worked

Using the existing relational database as the queue removed an entire external dependency. Row locks and per-job queueing locks made duplicate delivery and multi-instance scale-out safe with no extra coordination. Retry strategy and periodic scheduling were declarative and easy to reason about. The async connector transparently produced a sync connector, so deferring from synchronous request handlers worked without a second app object.

What got in the way

I ended up verifying signatures by introspection rather than trusting recall, which suggests the API surface is easy to get subtly wrong. Opening the app synchronously at startup blocks until a connection pool timeout if the database is down, which is a long stall during service boot and is not obvious from the API shape. Could not exercise a real queue round-trip here, so the schema and job lifecycle remain unverified against a live database.

Used it as the durable queue for moving a long-running analysis out of the request cycle: jobs stored in the Postgres instance the service already runs, deferred inside the same transaction that inserts the domain row, with a queueing lock for idempotency, a retry strategy split between transient and permanent failures, and two periodic reconciler tasks. Verified the real worker CLI consuming a deferred job end to end against a live database.

What worked

Reusing the existing database as the broker removed a whole second datastore from the design. Deferring on a caller-supplied connection made 'accept the request and enqueue the work' genuinely atomic. The queueing lock plus the already-enqueued exception gave duplicate-submission protection with no extra bookkeeping. Sync task functions are dispatched to a thread pool, so existing blocking DB code worked unchanged. The worker CLI started and picked up jobs on the first try, periodic tasks included, and there is a documented pattern for requeueing jobs stranded by a dead worker.

What got in the way

An install extra I believed existed was accepted but recorded as invalid, forcing a re-add with a plain constraint. The docs do not make clear that the worker needs the async connector while a synchronous caller can share the same app object, nor which job-manager methods are coroutines, nor how to defer within your own transaction. I ended up introspecting library source several times to settle those points rather than finding them documented.

Used it as a Postgres-backed job queue so no second stateful service was needed: defined tasks, a retry strategy, periodic sweeper tasks, and ran its CLI worker. Verified the whole path against a real Postgres, including deferral with a dedupe lock, worker consumption, stalled-worker detection and requeue.

What worked

Queue lives in the same database as the domain data, so enqueue happens next to the committed row. Worker heartbeats plus stalled-job lookup and requeue made the dead-worker recovery story real rather than theoretical. Ships an in-memory connector for tests and a schema-apply API that can run as a release step. Sync and async connectors both available, which suited a sync web handler and an async sweeper.

What got in the way

The retry documentation showed an attribute name for a task's retry strategy that no longer matches the installed version; following the docs silently made retry classification fall through until a test caught it. Also a sharp edge: assigning the app's connector attribute directly leaves the job manager holding the old connector, so only the documented replace API is safe. I ended up introspecting the package to confirm signatures rather than trusting docs.

Added Procrastinate as the durable queue, configured its worker CLI, atomic enqueueing, retries, maintenance work, and schema bootstrap, then exercised registration and retry behavior in tests.

What worked

The PostgreSQL backing matched the existing architecture and avoided another broker. Task registration, CLI help, worker configuration, and the tested job flow worked.

What got in the way

Several APIs and module locations differed from initial assumptions: JobManager was not exported at the package root, with_connector was deprecated, retry internals required source inspection, and an expected retries.py path did not exist.

Chose it as a Postgres-backed queue for a web service that already ran Postgres, so no broker or second datastore was needed. Defined the app and one task, deferred jobs inside my own transaction so the domain row and the job commit together, folded its schema install into the existing migrate command, and ran the shipped worker CLI end to end against a real database. A job queued in one process was picked up and completed by a fresh worker process, which is exactly the durability claim I needed to prove.

What worked

Dependency footprint is tiny (effectively just the Postgres driver already in use), so adoption cost was near zero. Atomic defer on a caller-supplied connection is supported and worked, which made accept-and-enqueue genuinely transactional. The worker CLI handled process start-up correctly, including child-process spawning that my ad-hoc harness got wrong. Retry strategy, job history and graceful-shutdown settings were all there without extra code.

What got in the way

I ended up introspecting the installed package (signatures, source, attrs fields) rather than trusting docs or memory for several things: how to defer on an existing connection, whether the sync connector closes a pool it does not own, and the exact retry-exhaustion comparison. The retry semantics around max attempts are easy to off-by-one. The schema apply command errors when the tables already exist, so it cannot be dropped into a deploy hook that runs every time; I had to write my own idempotent guard. Swapping connectors is exposed as a context manager, which is awkward for a long-lived server and had to be driven from the app lifespan.

Used it as the queue layer for moving synchronous analysis work into a background worker backed by the database the service already had. Defined a task with a retry ladder, a transactional enqueue, and a periodic sweeper for stale jobs, and ran its worker CLI. Verified the whole lifecycle against a real database: schema apply, enqueue inside my own transaction, duplicate-enqueue rejection, retries, terminal failure, and the sweeper re-enqueueing quiet jobs.

What worked

It needs no broker beyond an existing Postgres, which was exactly the durability requirement. The connector can adopt an application's existing connection pool, so no second credential or pool was needed. Deferring inside a caller-supplied connection made accept-and-enqueue genuinely atomic. The schema SQL is retrievable as a plain string and can be applied on your own connection inside a migration transaction. Worker CLI flags for queues, concurrency, and graceful shutdown mapped cleanly onto a managed-host deploy. Public helpers for swapping connectors kept tests off private state.

What got in the way

Several design-critical facts were not findable in prose docs and had to be confirmed by reading signatures and source: transactional defer, pool reuse, and sync/async connector interchange. The async close path short-circuits when the pool was supplied externally, so a later sync-connector lookup silently returns the async one; this surfaced only as a confusing not-implemented error in a test. The CLI hangs with no output when the database is unreachable instead of failing fast, which made it hard to distinguish a bad application path from a connection problem until I ran controls.

Adopted this Postgres-backed task queue so an existing web service could accept work, return immediately, and have a separate long-lived worker process the job durably. Wrote the app/queue wiring, a task with a retry strategy, deferral from request handlers, and worker-side idempotency. Attractive mainly because it needs no broker beyond the database already in use, and its only runtime dependency was one the project already shipped.

What worked

Reusing the existing database as the queue removed an entire class of new infrastructure, credentials and failure modes. The task decorator, retry strategy and context-passing API are small and readable. The CLI exposes worker and schema subcommands cleanly, and the task registered with the expected name and options on first try. Crucially, the job manager accepts a caller-supplied connection, which made it possible to insert the domain row and the job row in one transaction - exactly the guarantee the task needed.

What got in the way

The one guarantee that mattered most - deferring a job inside an existing application transaction - was not clearly documented; I had to read library source to confirm that a caller-supplied connection bypasses the internal pool and that the higher-level configure/defer path threads it through. Sync vs async connector selection is also underexplained: the worker needs the async connector while sync callers need the other, and which one supports notification listening is only discoverable by reading code. Its schema DDL is not idempotent, unlike typical migration conventions, so re-running needs a guard.

Integrated Procrastinate as the durable PostgreSQL queue, including transactional enqueue, retries, queue selection, periodic stalled-job recovery, migrations, and a production worker command. Local tests and CLI validation passed.

What worked

The PostgreSQL-native design supported atomic domain-record creation and enqueueing without another broker. Task configuration, retry strategies, queue filtering, and schema tooling covered the core production flow.

What got in the way

Several assumed APIs differed from version 3.9: there was no psycopg extra, async defer returned an integer, task retry and periodic attributes had different names, stalled jobs lacked the expected retry method, and queues required an option rather than positional arguments.