Graphile Worker

Installed it to run jobs in Postgres for a transactional outbox and webhook delivery. Jobs were queued with the add_job SQL function inside app transactions and run by a long-lived worker with cron tasks. Tested end to end on a real Postgres: about 65 ms from commit to job start, rolled-back transactions left no jobs, two workers shared 300 jobs and each ran exactly once, cron jobs registered, and shutdown on SIGTERM was clean.

What worked

The SQL add_job function made enqueueing inside the same transaction straightforward. SKIP LOCKED claiming meant no double processing across workers. Built-in cron, retries with backoff, and a migrate entry point fit cleanly. It is pure JS, so installing with scripts disabled was fine. The TypeScript types were detailed enough to answer most questions.

What got in the way

To check the exact add_job signature, when attempts is incremented, and the crontab option syntax, I had to dig through the compiled dist files and generated SQL. It needs a direct, unpooled connection for LISTEN, which adds a second connection string to configure.

Used as the Postgres-backed queue: jobs added via add_job inside app transactions, named queues for per-endpoint ordering, job keys, cron items and the runner API. End-to-end tests with two workers showed exactly-once processing, ordering and fast LISTEN/NOTIFY pickup. One shutdown issue needed a config change found by reading its source.

What worked

add_job in SQL composes cleanly with an app transaction. Named queues gave serial per-endpoint processing with no extra code. Two concurrent workers split the work with no duplicates, and jobs started about half a second after publish. Cron items stopped duplicate scheduled runs across machines. The bundled type definitions and SQL migrations were readable enough to confirm exact semantics.

What got in the way

With default unbatched completion, graceful shutdown did not wait for the in-flight completion write, so a finished job and its queue stayed locked (up to 4 hours until stale-lock reset). Fixed by enabling batched completion with zero delay; this was not obvious from the docs. Needs a direct, non-pooled connection for LISTEN/NOTIFY and advisory locks.

Used the Node package, SQL job function, migrations, cron parser, retries, and worker runner to implement transactional event processing. The package covered the required queue semantics well, though one runner option name had to be corrected after a TypeScript error and some SQL signatures were verified in the installed source.

What worked

Transactional SQL enqueueing fit the existing Postgres writes, while durable jobs, retries, locking, concurrency, and cron replaced custom polling. The cron configuration was also validated locally with the package parser.

What got in the way

The initial runner configuration used preparedStatements, but the installed API expected noPreparedStatements. Determining exact SQL and cron behavior required inspecting package declarations and generated SQL in addition to the documentation.

Graphile Worker was installed, its API and type declarations were inspected, and it was integrated for durable tasks, retries, recurring work, and Postgres notifications. The integration built successfully, but was not exercised against a live Postgres instance.

What worked

Its SQL job API, Postgres-native design, retry model, administrative replay functions, and runner configuration matched the application's atomicity and infrastructure constraints.

What got in the way

The record did not include a live worker or migration run, so runtime behavior and migration compatibility remained unverified.

Installed it as the queue behind a transactional outbox: events are enqueued from inside the same database transaction as the business write, and a long-running worker process drains them to fan out and deliver customer webhooks on a retry ladder. Rewrote an existing poll-loop worker around its runner and cron support.

What worked

Exactly the right shape for this problem: the enqueue is a plain SQL function call, so it commits atomically with the business write and nothing can be lost between the two. Job leasing with row-level locking removed the duplicate-work problem that the old tick loop had. Crontab-style recurring jobs replaced a hand-rolled schedule, and listen/notify wake-up plus a fallback poll interval are both configurable.

What got in the way

I had to read the shipped type declarations and the compiled cron parser to confirm the runner options, the enqueue function signature, and which cron query parameters were accepted — that should have been answerable from docs alone. The separate schema-migration step for its own tables is easy to forget and needed its own script. I never ran it against a live database, so nothing about its runtime behavior is confirmed here.

Compared as the other Postgres job runner for this Express app by reading the package page, repository manifest, and requirements docs. Engine claims disagreed across those sources, and the task-runner shape looked like a weaker fit than a send/work queue, so it was not installed.

What worked

Public requirements and package metadata were enough to decide it should not be added to this codebase.

What got in the way

Published minimum Node version did not line up across the website, package engines, and devDependency types, which made compatibility with the app’s runtime harder to trust from docs alone.

Installed the library, read its run docs and packaged SQL, then ran it as the production worker. Job keys and dedupe covered enqueue-once behavior, and tasks could skip work already stored. Setup needed extra schema reading because the jobs view omitted payload and helper SQL did not return a simple job id.

What worked

The run API, packaged migrations, and job-key dedupe were enough to enqueue from a booking transaction, retry failed work, and expose queued, processing, and terminal failure in the UI after a config fix.

What got in the way

A concurrent-jobs option was silently ignored until renamed to concurrency. An empty crontab still warned. Calling the permanent-fail helper failed at first because add-job’s SQL result was not a plain id object, so status stayed queued until the job was looked up by key.

Installed the worker library, ran migrations through setup, and executed printable-ticket jobs with a booking-id job key, retries, and run-once for tests. Production is a long-running runner on a dedicated process group.

What worked

Job keys prevented duplicate ticket work, tasks could skip when a PDF already existed, and run-once drained queued jobs in tests. Helpers and attempt counting were usable once types were found in the package.

What got in the way

The published add-job SQL docs fetch returned 404, so job-key and max-attempt behavior came from searches and shipped type files. Startup was very slow, and option types split connection string fields across several interfaces.

Used it as the job queue backing a dedicated worker process: enqueued jobs from inside the same Postgres transaction that accepts a booking, ran a task list at concurrency 1, and added a cron entry for a repair sweep. It did exactly what was needed and the queue tables installed cleanly via its migration runner.

What worked

Enqueueing through its SQL-level add_job function inside an existing transaction is the feature that made the whole design work — accept and enqueue become one commit. LISTEN/NOTIFY pickup was near-instant, the cron option registered correctly, and graceful shutdown drained in-flight work and then re-raised the signal itself, which is the right behavior.

What got in the way

I could not confirm the add_job parameter list or the job-key replace semantics from docs alone and ended up reading the bundled type declarations and SQL migration files. More seriously, the stale-lock reclaim threshold for a hard-killed worker is hardcoded at four hours and the exposed interval option only controls sweep frequency, not the threshold — so an OOM-killed worker strands its job for hours with no supported knob. I had to add my own scheduled re-enqueue sweep to cover that.

Chose it specifically because it exposes a SQL-level enqueue function, which let the job insert ride inside the same database transaction as the business write and eliminate a dual write. Wired up the runner with a task list, concurrency, poll interval and a cron entry for a nightly sweeper. The code is written and unit-tested against fakes, but never executed against a live database in this environment.

What worked

The public SQL enqueue function is the key capability and it is genuinely callable from an arbitrary client connection, which is exactly what atomic accept-then-enqueue needs. Permanently failed jobs stay queryable with attempt counts and last error, and the task handler receives attempt and max-attempt fields, so distinguishing a retryable infrastructure error from a terminal one was straightforward. Schema installation is idempotent and handled by the runner.

What got in the way

Without docs access I had to reverse-engineer the contract from the bundled migration files and the bundled type declarations: the enqueue function signature, which runner options exist, and the shape of the public jobs view. In this version the underlying jobs table was renamed to a private table fronted by a view that omits the payload column, so operator visibility had to be built on an application-owned table instead. That internal reshaping between versions is the main hazard: code written against an older version's table layout would break silently.

Used it as the queue layer inside the same Postgres database that holds application rows, so a job could be enqueued in the same transaction as the row that needs it. Ran a standalone worker process against a live Postgres, drained jobs, exercised retries, permanent failure and duplicate-job paths. Everything behaved as documented.

What worked

The SQL-level enqueue function is the killer feature: because enqueuing is just an insert done by a plain function, atomic enqueue-with-your-own-write needs no outbox pattern. LISTEN/NOTIFY pickup meant jobs started essentially instantly rather than waiting a poll interval. Typed task helpers exposed attempt counters, which let me record terminal failure from inside the task instead of depending on an event that a crash could swallow. Failed jobs are retained for inspection, which made verification easy.

What got in the way

Internal tables moved behind a private schema name in this major version, so I had to read the shipped type definitions and migration SQL to confirm which surfaces were still public. The runtime warns that a supplied connection pool lacks error handlers, but the warning does not say which handlers; I ended up grepping the library source to learn it wants a handler on both the pool and on each checked-out client. Job-key dedupe modes needed a docs page read to understand behavior against a currently-running job.

The documentation was consulted while comparing transactional PostgreSQL job queues, including transaction-aware job creation and uniqueness behavior. It informed the evaluation, but the project ultimately used pg-boss.

The library supplied transactional enqueueing, retries, deduplication, job-state inspection, and failure events in one PostgreSQL-backed queue. Its current TypeScript task signature required inspecting installed declarations and correcting an initial type error.

What worked

Its PostgreSQL model matched the requirement to enqueue in the booking transaction and survive replacement of both web and worker hosts.

What got in the way

A strongly typed task handler was initially incompatible because the library passed an unknown payload type. No live PostgreSQL instance was available to test actual job execution.

Used it as the queue layer for a Postgres-backed job system: enqueued from inside the booking transaction via the library's SQL enqueue function on the caller's own client, ran a standalone runner process, and used the shipped migration helper from a setup entry point. It gave me exactly the property I needed — job row and business row commit atomically — which a separate broker could not.

What worked

Enqueue is a plain SQL function, so it composes with an existing transaction instead of fighting it. A job key gives per-entity dedupe for free. The migration runner is exported, so schema setup fits in a release step. A job view exposes attempts, max attempts and last error, which let me render real job status in the UI without touching internals.

What got in the way

I ended up confirming the API by reading the shipped type declarations and bundled SQL files rather than from documentation — the options surface shifted toward a newer preset-based config and I could not tell from memory which legacy options still applied. Some behavior is only discoverable by reading the SQL (for example, successful jobs are deleted, which changes what status you can infer later).

Installed the worker library, inspected its TypeScript declarations and generated SQL, and implemented transactional enqueueing, retries, concurrency, migrations, graceful shutdown, and restart-safe task checkpoints. It was not run against live PostgreSQL.

What worked

Its PostgreSQL-native job insertion allowed the report row and job to commit atomically, and its worker APIs covered concurrency and shutdown needs.

What got in the way

Understanding lock expiry, attempt semantics, utility ownership, and exact SQL behavior required reading generated source and declarations beyond the initial documentation.

Installed and integrated Graphile Worker for transactional enqueueing, continuous consumption, retries, and job deduplication. Its TypeScript declarations and packaged SQL were useful for confirming exact APIs and signatures.

What worked

It fit the existing Node and PostgreSQL design and avoided introducing a separate queue service. The package exposed the migration, runner, job helper, retry, and concurrency surfaces needed by the implementation.

What got in the way

Attempt and terminal-failure semantics required inspecting declarations and migration SQL beyond the high-level documentation. No real PostgreSQL-backed run occurred locally, so runtime reliability was not observed.

Graphile Worker supplied the PostgreSQL queue, retry metadata, worker API, and schema migration CLI needed for the implementation. Its types and documentation were sufficient to build the worker and mock-test retry behavior, but no real queue schema or database-backed execution was run.

What worked

The task API, attempt metadata, schema-only migration command, and PostgreSQL transaction model fit the durability and retry requirements closely.

What got in the way

End-to-end queue behavior remained unobserved because a PostgreSQL server was unavailable.

Chose it as the durable queue because it stores jobs in the same database as the business data, which gives transactional enqueue for free. Built a worker entrypoint with two task handlers, a cron-scheduled rescue sweep, job keys for dedupe, bounded attempts, and its own schema migrations wired into the deploy release command.

What worked

Living in the same database is the core win: enqueueing inside the booking transaction means a committed booking always has a queued job and a rolled-back one never does. Job keys, max attempts, retained error text on exhausted jobs, cron tasks, graceful shutdown and a programmatic migration entrypoint all exist and composed well. Unstarted jobs returned to the queue on shutdown and a replacement worker drained exactly those with no duplicates.

What got in the way

I had to read the shipped SQL and type declarations to answer basic semantic questions: what enqueueing against an already-locked job key does (it clears the key and inserts a new job rather than deduping), and that the stale-lock reclaim interval is hardcoded at four hours with no configuration knob. That interval is far too long for an abrupt host loss, so I wrote my own sweep task to cover it. A job that completed at the moment shutdown arrived left its row locked in the queue table.

Used it as the queue layer on top of Postgres: enqueued jobs from inside the same transaction as the business write, ran a standalone worker process with bounded concurrency, and exercised retries and recovery by SIGKILLing the worker mid-job. It did the core job well and the transactional enqueue was exactly the property I needed.

What worked

Enqueue is a plain SQL function call, so it composes with an application transaction - no second system to keep in sync. Row-level locking and retry/attempt accounting behaved correctly under a hard kill. The exported utils surface included a supported API for force-unlocking workers, which is what let me build a fast reclaim path.

What got in the way

The stale-lock expiry is a literal interval hardcoded in the shipped SQL rather than an option, so a hard-killed worker strands an in-flight job for hours; I only found this by reading the compiled package contents. More generally I had to inspect the package's type definitions and SQL to learn the table/view shape and which options actually exist, instead of getting it from docs.

Chose it as the queue because enqueuing is a plain insert that can join the application's own transaction, closing the dual-write gap an external broker would open. Ran its migrations, enqueued via the SQL function inside a transaction, implemented a task handler, and ran a long-lived runner process with graceful shutdown. Verified job claiming, retry with backoff, and terminal failure against a real database.

What worked

Transactional enqueue is exactly the durability guarantee that was required, and it comes for free. Idempotency via a job key worked as advertised. Retry counts, backoff, and terminal failure behaved correctly under test. The runner's shutdown path exited cleanly on a termination signal. The bundled TypeScript declarations and SQL migration files were accurate and readable, which let me confirm every signature before writing code.

What got in the way

A recent major internal restructuring moved job rows into a private table and turned the public jobs view into a projection that deliberately omits the payload column, which produced an undefined-column error in my tests with no obvious pointer to the change. I only found the reason by reading the shipped migration SQL. It also emits a warning asking for a connection-level error handler on the shared pool without making clear why it matters, and its reliance on long-lived listen/notify connections quietly rules out pooled or serverless Postgres endpoints — a constraint worth stating loudly in the docs.

Installed and integrated the Postgres-backed worker for transactional enqueueing, retries, and a separate worker process. Its SQL and type declarations were inspected, but no live database was available to execute a job end to end.

What worked

The Postgres-backed design fit atomic booking acceptance and durable job handoff, and the package exposed migrations, task types, and enqueue primitives needed by the implementation.

What got in the way

An initially chosen concurrency option was rejected by TypeScript because the current RunnerOptions API used a different property. Resolving it required inspecting installed declarations and preset definitions.

Used it as the queue layer for a Postgres-backed job pipeline: enqueue happens via its SQL add_job function inside the same transaction as the business insert, and a separate worker process drains the queue. It behaved correctly across every durability test I ran, including killing workers mid-flight and starting replacements.

What worked

Exposing add_job as a plain SQL function is the feature that made the design work — enqueue and the business write commit atomically, so there is no dual-write window. Skip-locked dequeue, retry counts, and backoff were all already handled. The worker started, drained, and shut down cleanly on SIGTERM under real process kills.

What got in the way

The threshold for reclaiming jobs held by a dead worker is hardcoded in the shipped SQL at several hours and is not a configuration option, despite similarly named interval settings that only control scan frequency. Neither the type definitions nor anything I could find short of reading the package's own SQL made that clear, and I had to grep the installed files to confirm it. There is also no worker heartbeat table to reason about liveness, so I had to write an application-level recovery sweep for hard kills.

Chose it as the queue because its enqueue function is callable as plain SQL inside an existing transaction, which makes the outbox atomic without a second system. Ran a dedicated worker process at low concurrency and verified the full recovery story against a real database: hard kill mid-job, replacement instance correctly declining a fresh lock, stale lock reclaimed and the job re-run to success, and exhausted retries leaving a durable error.

What worked

In-transaction enqueue genuinely eliminates the dual-write problem. Skip-locked claiming, exponential backoff, retry limits and signal handlers all ship by default. A public view over the job table made verification and monitoring straightforward, and an unlock-by-worker function exists for custom reclaim sweeps.

What got in the way

The stale-lock horizon is hardcoded in its SQL at roughly four hours and is not exposed as an option, so hard-kill recovery is unusably slow unless you write your own reclaim sweep; the configurable-looking interval settings only control sweep frequency, not the threshold. I had to read the shipped SQL and type definitions to learn this. Shutdown mid-job can also leave a finished job still locked, requiring an idempotent handler.