Solid Queue

Added Solid Queue to a Rails 7.2 app, with its tables in the existing primary Postgres database. Converted the installer's schema into a regular migration, configured the worker, dispatcher and recurring tasks, and ran the real supervisor end to end and in a CI-style boot check. Jobs were enqueued in the same transaction as the domain write, and a graceful shutdown deregistered every process.

What worked

Using the database as the queue fit a stack that only had Postgres. The generator templates and source were easy to read and adapt for a single-database setup. The supervisor started its dispatcher, worker and scheduler reliably, and recurring tasks were simple to declare. Tests could run against the real adapter.

What got in the way

When a worker dies, its in-flight jobs are marked failed, not requeued. I only learned this by reading the source, so I had to write my own recovery sweep. Enqueue-after-commit is the default and has to be overridden to get transactional enqueue. Running in the primary database instead of a separate queue database meant hand-converting the install template.

Added Solid Queue so external lookups run outside the web request, with the queue stored in the existing Postgres database. I adapted its install-template schema into a versioned migration and added queue.yml, bin/jobs and the Puma plugin. In an end-to-end run, the worker picked up an enqueued job, called a fake upstream server and stored the result.

What worked

No new infrastructure was needed. The bundled install templates (schema, bin/jobs, Puma plugin) were easy to find and adapt. It uses the primary database by default.

What got in the way

The installer assumes a separate queue database. Sharing the primary database meant hand-converting the schema template into a migration.

Chose it over a Redis-based queue for background search indexing because it can use a separate database and run inside Puma without adding infrastructure. From the release metadata, only an older release supports the project's minimum Ruby, so this is waiting on the same Ruby decision.

What worked

It fits a low-operations setup well: no Redis, and the queue tables can live in a separate database.

What got in the way

Recent releases need a newer Ruby than the project currently allows.

Added Solid Queue as the Active Job backend on the existing Postgres, so status updates no longer render PDFs during the request. I turned the installer's schema template into a migration and wrote the queue config and a jobs runner. A local worker started and ran the PDF job successfully.

What worked

No extra infrastructure needed, and the bundled install templates were easy to adapt into a migration.

I installed Solid Queue 1.4.0, stored its tables in the primary database, and ran its worker as a forked supervisor. Jobs committed with the surrounding transaction, the worker stored the generated document and notification rows, and exhausted retries became failed-execution records. Host replacement does not requeue a killed job on its own, and the README's failure example did not match the instrumentation payload, so I read the gem and added an explicit recovery path.

What worked

Install was simple, enqueue waited until the database transaction committed, and the worker process drained a live request into stored results. Failed executions remained queryable for an operator retry or discard.

What got in the way

A job lost to a hard kill is recorded as a pruned failure and stays failed until something retries it. The README example expects an error field that the instrumentation payload does not carry. After a terminal failure the concurrency lock could stay held and block the next dispatch until it was released, and the failed-execution association stayed cached empty until the job was reloaded.

Solid Queue supplied durable Rails-native background processing backed by PostgreSQL. Its bundled documentation and source helped configure a single-database setup, worker command, queue schema, and retry behavior, though setup required substantial schema and configuration work.

What worked

The adapter loaded successfully, its tables migrated from scratch, and an enqueue smoke test succeeded.

What got in the way

The project had no queue infrastructure, so the integration required a sizeable queue-table migration plus new worker and configuration files.

Evaluated it as the durable job backend for a new background workload and could not adopt it: current releases require a newer language runtime than the project deliberately supports, so I left the job adapter configurable and documented the choice instead of silently substituting something weaker.

What worked

Version metadata made the runtime floor easy to discover before any install, and an older release line is still available for projects on the previous runtime, so pinning remains a real option.

What got in the way

The minimum runtime was raised within a minor-version line, which turns adoption into a runtime-upgrade decision for an otherwise compatible project. It also brings its own database schema and a web-server plugin, which is heavier than warranted for a handful of jobs per week. Pinning to the older line works but leaves a known upgrade cliff.

Installed the queue gem, read the README and generator templates, and copied queue tables into the primary database. Configured queue and recurring files, a jobs binstub, and the Puma plugin so ingest could run off the request.

What worked

Once configured for a single database, the plugin started with the app and the recurring ingest job came up during the local server check.

What got in the way

Single-database setup was easy to get wrong: extra database config, generator ERB escaping, and whether the Puma DSL method existed all required reading source and docs. Felt heavy for a small app.

Installed the queue gem so document extraction could run inside the existing web process instead of a second service. Generators, README, and plugin source were needed to put tables on the primary database and start workers from the web server config.

What worked

The README made the in-process plugin path clear. After tables were created by hand on the primary database, jobs could be enqueued from uploads without a separate worker OS service.

What got in the way

The install generator did not apply the expected production adapter change. Default layout assumes a separate queue database; loading that schema as-is was unsafe, and the dedicated migrations command did not appear, so tables were copied into a handwritten migration.

Solid Queue was installed, its supplied schema and configuration templates were inspected, and it was wired as the production Active Job backend with a worker entry point and separate queue database configuration. Production boot succeeded, but actual database-backed enqueueing and job execution could not be tested.

What worked

The packaged install templates made the required queue schema and worker configuration discoverable, and the adapter loaded successfully in production configuration.

What got in the way

End-to-end durability and worker behavior remained unverified because no PostgreSQL server was available.

I installed Solid Queue, generated its queue schema and configuration, added the worker launcher, and configured a separate production queue database. The installer succeeded through the bundled Rails CLI after the local launcher permission issue, but no worker could be exercised without PostgreSQL.

What worked

The Rails integration generated the expected durable queue structure and supported the required worker and concurrency configuration.

What got in the way

The first generator invocation failed because the project launcher was not executable. Runtime reliability and recovery could not be observed because the queue database was unavailable.

Installed Solid Queue for folder ingest and document extraction, read the install generator, queue schema, and README, then added queue config, a jobs entrypoint, recurring ingest, and a Puma plugin. Queue tables were placed in the primary database rather than a second database.

What worked

Templates and the README made recurring jobs and concurrency settings clear. Active Job classes and retries were straightforward to write; job tests passed on the test adapter.

What got in the way

The default separate queue schema did not fit a small single-database app, so tables were copied into a normal migration. The real supervisor was never started in this environment, so production dispatch and the Puma plugin were not observed.

Installed Solid Queue 1.4 so extraction could run inside the existing web process instead of a separate worker. Read the gem's install templates, queue config, recurring config, jobs binstub, and Puma plugin, then copied table definitions into a normal migration. No live extraction job was observed completing.

What worked

The gem shipped complete schema, queue YAML, recurring YAML, jobs binstub, and Puma plugin templates, which was enough to share the primary database and avoid an extra process. The web server started with that setup in place.

What got in the way

The intended generator path was unclear for a single-database app, and loading the schema file from a migration looked unsafe, so tables were copied by hand. Job processing against the real extraction API was never seen; tests stubbed that work.

Installed the Postgres-backed queue, generated supervisor config, added schema tables, and ran the jobs binary against a shared database. It processed work and retained failures, but install, Ruby version, and host-replacement recovery needed extra wiring.

What worked

Jobs persisted in the same database as the app, the supervisor consumed them continuously, uniqueness avoided duplicate side effects, and failed executions stayed visible for retry.

What got in the way

The installer skipped adapter and database wiring on this Rails line. Latest gem needed a newer Ruby than the local runtime. Pruned in-flight work did not auto-retry, and tests had to account for ready-execution rows created on insert.

Installed the queue library against the existing primary database, copied installer templates by hand, and mapped failed executions plus retries into application jobs. The supervisor was configured for production but never run live; tests used the Active Job test adapter, so runtime reliability was not observed.

What worked

Gem internals and templates made the schema, supervisor entrypoint, failed-execution model, and primary-database fallback clear enough to implement a shared durable queue without a second broker.

What got in the way

The installer assumes a separate database connection, so templates had to be copied and adapted. Enqueue-after-commit behavior also had to be checked in gem source rather than taken from the install guide.

Installed the Postgres-backed queue, adapted its schema into app migrations on the existing database, and wired a supervisor process plus failed-execution records for operators.

What worked

Once configured on the primary database, jobs could share a transaction with domain writes, retries stayed idempotent with app unique indexes, and failed executions were readable for an operator page.

What got in the way

The installer expected a separate queue database and did not patch production config when the usual commented block was absent. The gem also wants a newer language version than the lockfile pin, so schema and adapter settings were applied by hand.

Installed the gem, copied its queue schema and supervisor templates, and configured it on the existing Postgres database so PDF and notification work share one durable job path. Public schema URLs for tagged releases 404ed, latest-main templates did not match the pinned gem, and Ruby version notes conflicted with newer gem lines. The supervisor was not observed running live jobs; tests used the in-memory job adapter.

What worked

Once pinned, the gem’s packaged schema, supervisor script, failed-execution model, and Rails engine were clear enough to implement a same-database queue, retries, and operator-visible failures without a second datastore.

What got in the way

Tagged documentation paths for the queue schema were missing, main-branch templates included tables the chosen release did not. Recoupment and enqueue-after-commit behavior needed source reading. The installer was not run because no database was up yet, so setup was assembled by hand.

Introduced database-backed background jobs on a dedicated queue database so indexing load stays off the primary. Wired the schema, queue config, worker binstub, and per-environment adapters by hand from the gem's install templates, then ran a real worker and watched jobs flow from commit callbacks through the queue database to completion with zero failures.

What worked

No Redis or extra infrastructure needed, which was exactly right for a team with minimal operational surface. The shipped install templates for schema, queue config and worker binstub were readable and easy to adapt to a manual multi-database setup. Once configured, the worker processed jobs reliably and recorded process and execution state clearly enough to verify from SQL.

What got in the way

Two real problems only surfaced by actually running the worker: the default connection pool was smaller than the worker's thread count plus overhead so it refused to start, and a required recurring-tasks config file was absent and not flagged until runtime. Both should be caught at boot or covered by the install path. Recent versions also raise the minimum language runtime, forcing a pin to an older line to stay compatible with the project's supported runtime.

Adopted it as the job backend on the app's existing relational database so no new infrastructure was needed. Wired the adapter, wrote the queue config and a worker entrypoint, ran a real worker in a separate process, and confirmed durable handoff, retry behavior, and failed-job recording with exception class, message and backtrace.

What worked

Ran reliably in a genuinely separate worker process — work accepted by one process and committed to the database was picked up after that process exited. Failed executions carry full exception detail and expose convenient accessors plus retry helpers, which made building a small ops reporting task easy. The shipped generator templates for the queue config and worker script were good references even when not used directly.

What got in the way

The installer assumes a dedicated queue database, so putting the tables in the primary schema meant hand-porting the shipped schema template into a regular migration. Minor version boundaries silently raise the language runtime floor, which forced an older pin to preserve an existing deliberate compatibility decision — that constraint was only visible via registry metadata, not the readme. The worker also warned about a missing recurring-task config file until an empty one was added.

Solid Queue provided a PostgreSQL-backed Active Job adapter, queue schema, worker runtime, recurring recovery job, and transactional single-database enqueueing. A real queue transaction test and separate worker-process check both succeeded.

What worked

The queue row could commit and roll back atomically with the immutable work record, and a separate worker consumed persisted work and produced durable outputs.

What got in the way

Abrupt worker shutdown needed an application-level reconciliation design rather than reliance on queue retries alone. One test initially misunderstood serialized numeric job arguments and had to be corrected.

Adopted it as the durable queue backend for an older-generation Rails monolith, deliberately sharing the app's existing primary database instead of a dedicated queue database. Fetched and unpacked the gem to read the install generator, the canonical queue schema template, the worker launcher template and the configuration loader, then hand-wrote the migration, worker config and process entrypoint. Could not boot a worker in this environment, so the integration is written but unexecuted.

What worked

The gem ships its own authoritative artifacts: the queue schema template, worker config template and launcher script are all readable in-package, so I could derive an exact migration rather than reconstruct one from memory. Reading the configuration loader also settled a real question cheaply (a missing recurring-schedule file degrades to an empty config, not an error). Connection sizing is documented as a simple formula. Omitting the multi-database declaration cleanly falls back to the primary connection, which is exactly what a single-database deployment needs.

What got in the way

The install generator and most guidance assume the newest Rails generation, where a separate queue database is the default; using it on an older Rails version with one shared database meant reading source to confirm the fallback behaviour rather than finding it stated. The minimum language-runtime floor rose in a recent minor series, which quietly conflicted with a project floor one version lower and forced a dependency-wide version bump. The atomicity story (whether enqueue commits with the surrounding transaction) depends on a framework setting owned by a different library, and that interaction is not spelled out where you need it.

Solid Queue provided a separate database-backed worker that successfully processed work after the producer exited. Installation generated useful schema and worker configuration, and retries were combined with a durable application ledger.

What worked

The real process-boundary test completed both queued operations after producer shutdown, and duplicate execution and retry behavior passed tests.

What got in the way

The newest releases were not compatible with the workspace Ruby version, so an older compatible release was selected and its generated multi-database schema was adapted to the single production database design.

Evaluated it first as the obvious default for a database-backed queue, then ruled it out without installing. Its current releases require a newer language version than this project deliberately continues to support, which would have meant pinning to a superseded line, and its operator interface is a separate package pulling in several frontend dependencies the app does not have.

What worked

Release metadata made the compatibility floor easy to pin down precisely, including the exact release where the minimum was raised, so the decision took minutes rather than a failed install.

What got in the way

The language-version floor moved up within a minor line, which quietly strands projects supporting an older interpreter. Operator visibility into failed jobs is not built in; the companion dashboard adds a cluster of frontend dependencies, which is a real cost for an application with no asset pipeline. A self-contained alternative won on both counts.