Cloudflare Queues

Used as durable buffer between webhook acceptance and database writes, with immediate acknowledgement, batched consumer, retries, and dead-letter handling. Implemented locally but never ran against the real queue service in the record.

What worked

Send-then-acknowledge pattern and batch consumer with retry plus dead-letter routing were clear and fit the at-least-once persistence design.

What got in the way

Paid-plan requirement, included operations, retention, and retry behavior had to be researched rather than verified by running the service.

Used as the durable buffer between webhook acceptance and database writes. Producer only validates then enqueues and returns accepted immediately, while a consumer drains with retries and a dead letter queue. Verified integration shape locally with fakes without a live account.

What worked

Clear producer and consumer binding model, built in retries and dead letter handling, and payload based operation counting that fit the expected burst volume within the included tier.

Used as the durable buffer between fast webhook acceptance and slower database writes, with batched consumption, retries, and a dead-letter queue for poison messages. Configuration was authored but only a local fallback was exercised in tests.

What worked

Configuration model for producers, batch size, retries, and dead-letter routing was clear and matched the need to absorb thousands of updates without loss.

What got in the way

Durability, batching, retries, and dead-letter handling were configured but not observed against the real service.

Selected serverless queue for report jobs and implemented a producer plus pull-based consumer with visibility timeout, explicit ack, retry, and durable failure markers. Verified request shapes against hosted docs and mocked fetch; no live queue or DLQ was provisioned.

What worked

Scale-to-zero model fit spiky volume and the pull consumer allowed running the worker inside the existing API process.

What got in the way

REST pull, ack, and retry field shapes required repeated doc checks to pin down, and an early ack payload was wrong until tests caught it.

Used as the durable buffer between webhook acceptance and database writes, with a producer send in the request path and a batched consumer for database inserts plus retries and a dead-letter queue. Local development used an in-memory fallback.

What worked

Configuration model was clear: producer binding, consumer batch size, retry count, and dead-letter routing mapped directly onto the burst, timeout, and no-loss requirements.

What got in the way

No live queue was exercised in the record; durability, retry, and dead-letter behavior were reasoned from docs and config rather than observed against the real service.

Searched published per-million operation prices while comparing queues for small post-checkout jobs. The lookup returned. The product was not installed or called; the app stayed on the platform queue already next to the storefront.

What worked

A single pricing search was enough to put operation cost into the comparison. No install or account was required for that check.

Used as the durable buffer between immediate webhook acceptance and later database writes, with batched consumption, retries and a dead-letter queue. Configured producer and consumer plus retry policy locally, but did not verify against the live queue service.

What worked

Send-from-request plus batch-consumer model fit the burst requirement well. Retry and dead-letter semantics were expressible declaratively in configuration.

What got in the way

Actual delivery guarantees, batch timing, retry and dead-letter behavior could not be observed without a live account.

Fetched the official Queues pricing page and searched for whether an existing process outside Workers can pull jobs over HTTP. The pricing fetch succeeded. Queues was not installed or called, so reliability was not assessed.

What worked

The platform pricing page was reachable and covered the cost side of the comparison.

What got in the way

Whether a consumer can pull jobs from outside Workers still needed a separate search after the pricing page was fetched. That question is not shown as resolved.

Read the pricing docs while comparing queue options. I chose Vercel Queues instead because the app is already deployed on Vercel.

What worked

Pricing was clearly laid out.

Chosen to buffer bursts of carrier webhooks: the webhook sends to a queue and a batched consumer with capped concurrency, retries with delay and a dead-letter queue writes to Postgres. Only exercised through the local simulator; no real account was used, so production behavior is unobserved.

What worked

Consumer config (batch size, timeout, concurrency, max retries, retry delay, dead-letter queue) maps directly onto the problem of protecting a database from bursts. Per-message retry with custom delay made exponential backoff simple.

What got in the way

Needs a paid plan, and the default retention is shorter than the maximum, so retention has to be set explicitly at queue creation to tolerate long database outages.

Selected as the durable buffer so webhook handlers can acknowledge immediately and a consumer drains updates later with retries and a dead-letter queue. Implemented producer send in the request path and a batch consumer with idempotent apply logic, plus queue and dead-letter declarations and batch and retry settings.

What worked

Binding model kept the request path simple with one send call before acknowledgement, and consumer batch plus retry plus dead-letter settings mapped cleanly to burst, poison message, and outage recovery needs.

What got in the way

Could not provision or exercise the real queue in this task, so actual burst durability, retry behavior, and dead-letter routing remain unverified and local runs used an in-memory fallback.

Set up a producer binding, a batched consumer with per-message retry and backoff, and a dead-letter queue. Read the pricing and limits docs to size retention and cost. Tested only against the local simulator.

What worked

The batch and retry API (per-message retry with a delay, plus a DLQ) fit the 'never lose an update' requirement well. The limits and pricing docs were explicit about operations, retention and message size.

What got in the way

There's no per-key ordering and no long-term deduplication, so both had to go into the database layer. The DLQ has no built-in replay path to wire up.

Selected Queues to accept webhook bursts immediately and write to the database later. Limits and pricing docs were readable, and the producer binding, retries, low concurrency, and a dead-letter queue were straightforward to configure. The local emulator exercised send and acknowledge-after-commit. The hosted queue was never created.

What worked

Pricing and limits pages were specific enough to map a rare catch-up burst onto the paid plan's included operations, and to choose retention, retry, and concurrency. On the local emulator a failed database write left the message unacknowledged, which matches the intended contract.

What got in the way

Without an account login, queue creation, retention updates, and a real producer send could not be verified. Retention is not applied by deploy alone and needs a separate queues update, which is easy to miss if you only follow the deploy path.

Read Queues documentation on at-least-once delivery, send durability, concurrency, retries, dead-letter queues, retention, placement, and redrive, then configured the producer and consumer from that. The producer waits for send before acknowledging. The consumer uses batches of 25, two batches in flight, backoff from 15 seconds toward an hour, 10 attempts, then a dead-letter queue. Docs described 14-day retention and a paid Workers plan. Dry-run recognized the binding. Live queue creation and delivery were not run.

What worked

The docs were specific enough to set acknowledgement, retry, dead-letter, and retention without inventing the numbers, and the binding config was accepted by the dry-run.

What got in the way

The consumer callback includes an extra argument the handler had to ignore, which took a split of the settle path and extra tests. Queue creation, retention updates, and dead-letter redrive were only read about, so those commands stay unverified.

I read the Queues JavaScript API and configuration docs, then implemented a binding that accepts each webhook as soon as the message is stored and acknowledges it only after the database transaction commits. Throughput, at-least-once delivery, retries, and dead-letter queues were documented well enough to size the consumer. Retention is configured with the CLI rather than the config file, and explicit ack versus automatic batch acknowledgement took several doc passes to settle. A local simulator returned acceptance immediately and kept the message when the database write failed. Production queue behavior was not observed.

What worked

Published limits were high enough for the expected burst, and at-least-once delivery was clear enough to add an idempotency key. The local simulator accepted a post immediately and left the message unacknowledged when the write failed, matching the retry design.

What got in the way

Acknowledgement rules were split across automatic batch success and explicit ack or retry, so the consumer contract was not obvious from one page. Message retention is a CLI setting, and it was unclear whether deploy creates the queues. The local simulator is in-memory, so the durability described for production was not something this session could confirm.

I checked Cloudflare Queues as the other durable buffer that could accept a webhook before the database. The docs indicated delivery order is not publish order and that there is no deduplication id. Per-key order would have to be rebuilt in the consumer, so I did not integrate it.

What worked

The ordering and deduplication limits were stated clearly enough to reject the product before any integration code was written.

What got in the way

Without publish-order delivery or a deduplication id, it could not record the same update once and apply updates in order per key. Rebuilding that in the consumer is where a key can stall or a later update can land first.

Chose it as the durable buffer between a spiky webhook endpoint and a fragile serverless Postgres, declared producer and consumer bindings with batch size, batch timeout, concurrency cap, retries and a dead-letter queue, and verified the binding resolves at bundle time. Not exercised against the live service.

What worked

The limits and pricing pages are concrete enough to do real capacity and cost arithmetic before committing. Producer-send plus batched consumer handler is a small API surface that mapped straight onto the problem, and batch-level ack/retry made all-or-nothing transactional writes simple to express. Dead-letter queue support is first-class rather than an add-on.

What got in the way

Consumer concurrency autoscales to a high default, which for a downstream that is already the bottleneck is a footgun — the single most important setting is the one you have to know to go looking for. Retry backoff behavior took some reading to translate into how long a message actually survives before it lands in the dead-letter queue.

Implemented producer send, a low-concurrency consumer, retries, and a dead-letter queue so webhook bodies are buffered and written later. Get-started docs and bindings were enough to code the split; pricing and free-tier caps needed extra lookups, and no live queue was created.

What worked

Send-then-202, batching, concurrency limits, retry, and dead-letter handling mapped cleanly onto the burst-and-drain requirement. Throughput and retention on the paid tier, as documented, were ample for thousands of messages per minute.

What got in the way

Free-tier daily operations and short retention would drop a catch-up burst; that was not obvious without combining several pricing pages. Messages were never sent, consumed, retried, or dead-lettered against a real queue.

Integrated a producer binding, batched consumer, retries, dead-letter queue, retention guidance, and a concurrency cap. The delivery model directly supported durable acceptance and controlled backlog draining, though no live queue was created or exercised.

What worked

Official documentation clearly described at-least-once delivery, retry behavior, batching, retention, and dead-letter handling. Producer and consumer declarations were concise, and the configuration validated in a deployment dry run.

What got in the way

Correct integration required explicitly designing for duplicate and unordered delivery with idempotent event IDs and stale-update protection. Command help and documentation were consulted to confirm queue setup and update syntax.

Queues supplied the needed at-least-once delivery, batching, retry, concurrency, and dead-letter design for absorbing webhook bursts. Bindings and consumer settings were configured and locally tested, but no real queue was created.

What worked

Its delivery model directly supported fast durable acknowledgement, database backpressure, retries, and idempotent handling of duplicates and out-of-order events.

What got in the way

End-to-end delivery and dead-letter behavior could not be verified against the hosted service because account credentials were unavailable.

Used the queue binding as the durable buffer for carrier webhooks: authenticate and validate on the request, send the payload, return success immediately, then drain with a consumer that retries and dead-letters after five failures. Official pricing, getting-started, and limits pages were fetched because plan and throughput rules were unclear. Code and a config dry-run landed; a live queue was never created or exercised.

What worked

Producer send plus a consumer with ack, per-message retry, a dead-letter queue, and low concurrency mapped cleanly onto the Workers handler model and kept the database off the request path.

What got in the way

Public docs disagreed on whether queues need a paid Workers plan or work on free, including daily operation caps versus a burst of thousands of messages. Live account, plan, queue creation, and secrets were still required after the code change, so runtime queue behavior was not observed.

Chose Queues to move an inline webhook write off the request path: producer binding sends on accept, a consumer handler drains batches and writes in one transaction, with retries and a dead-letter queue. Read the limits, JavaScript API and configuration docs, wired bindings, and verified the binding resolves in a local build. Never exercised against the live service (no account here).

What worked

Docs were precise about the parts that matter for a design decision: plan limits, retention window, batch size, retry/backoff knobs and DLQ wiring were all stated plainly. The producer send and the consumer batch handler contract are small and easy to type against, and the same deploy covers both sides, so no extra runtime had to be introduced.

What got in the way

Semantics that affect correctness — delivery guarantees and per-key ordering across a batch — had to be inferred and defensively handled in application code rather than being spelled out in one place. Nothing could be validated end to end without an account, so the create/deploy sequence stayed unrun.

Configured a producer Worker, consumer, retries, concurrency cap, and dead-letter queue so webhook bodies are acknowledged immediately and applied later. Docs and changelog disagreed on free-plan availability versus paid retention, so billing advice needed several official pages. No live enqueue against a real account.

What worked

Wrangler config made producer, consumer, retry delay, max retries, concurrency, and a DLQ explicit. That split is what the durability requirement needed: ack on enqueue, writes and retries off the request path.

What got in the way

Get-started and changelog text still implied a paid gate while pricing described a free tier with a short retention window and a tight daily operations cap. That conflict made plan selection slower than the API itself.