Google Cloud Tasks

Used as the recommended queue for accepting publish clicks immediately and retrying portal, brochure, photo and email steps in the background with deduplication. Added the Python client library, built a production-plus-in-memory queue layer, wired queue path and worker auth settings, and covered single-delivery and retry behavior with tests.

What worked

Client library installed and imported cleanly. Task naming plus worker-side idempotency checks mapped well to the never-push-twice requirement, and the in-memory fallback made local testing practical.

What got in the way

Live service behavior was not observed; verification used the in-memory queue and fakes, so retry/backoff and duplicate suppression against the real service remain unverified.

Added the Cloud Tasks client library and implemented enqueue plus worker handling for an async publish flow with deduplicated tasks, idempotent portal writes, and queue-driven retries.

What worked

Enqueue API and task naming made duplicate-click suppression straightforward, and moving retries to the queue removed custom retry loops.

What got in the way

No live service call was made during the task; retry and delivery behavior was exercised through in-repo fakes only.

Installed the queue client library, implemented ID-only push tasks with local fallback, and verified payload construction and handler retry behavior with mocked tests.

What worked

Client helpers for queue paths and HTTP push tasks were clear, and fallback to the existing background worker kept local development working without credentials.

What got in the way

The first pinned library version did not exist and was rejected during install, requiring a downgrade to an available patch release.

Researched retry, queue and HTTP-target docs and implemented an enqueue plus worker-handler design with backoff as the retry mechanism and per-step persisted progress to allow resume without duplicate paid portal submissions.

What worked

Concepts for retries, deadlines, deduplication and ordered per-listing execution mapped cleanly to the need for fast accept, background work and never-send-twice behavior.

What got in the way

Documentation was spread across several pages and required repeated fetching to confirm retry parameters and routing behavior.

Installed the pinned client library, imported the tasks client module, and verified queue naming and HTTP target helpers while building a production queue plus in-memory queue for local use. Never called the live service; retries were exercised through the local queue and worker endpoint.

What worked

Install was smooth and the client API for queues, named tasks, and HTTP targets was clear to wrap behind a small abstraction.

Selected as the async queue for a long multi-step publish flow. Implemented named tasks for click dedup, HTTP delivery to the existing service, retries with backoff, and dead-letter handling. Live service was not exercised; tests used an in-memory queue.

What worked

Named tasks made double-submit handling simple. Retry, rate limiting, and dead-letter concepts mapped cleanly to burst and transient downstream failures without adding a server.

What got in the way

Live behavior, IAM, queue provisioning, and timeout interactions were not observed in the record.

Used Cloud Tasks with HTTP targets for accept-fast then work-later publishing, with backoff retries and a separate idempotent worker to prevent duplicate paid portal posts. Local tests ran against an in-memory queue; the hosted queue itself was configured but not exercised live.

What worked

Concepts mapped well to the problem: immediate accept, long worker timeout, exponential backoff with limited attempts, and at-least-once delivery paired with idempotent steps. Client library install was straightforward.

What got in the way

Live retry, dead-letter and queue tuning behavior could not be observed without credentials, so production retry behavior remains unproven.

Recommended and implemented the publish flow around a named deduplicated task queue with a same-service task handler, single-attempt portal steps, and backoff redelivery. Local verification used an inline and mocked queue only, so live service behavior was not observed.

What worked

Named per-listing tasks mapped cleanly to double-click deduplication, and task-level retries with backoff fit the long chain and transient renderer and portal failures better than in-request retries.

What got in the way

No live enqueue, delivery, IAM grant, or queue configuration was exercised in the record; deploy-time queue settings and permissions remained manual follow-ups.

Used the task queue SDK as the production backend behind a small queue abstraction, with an in-memory fallback for local development and tests. Implemented named tasks, delayed retries with backoff, idempotent worker handling, and ordered unpublish behavior. Local paths were fully tested; production queue behavior was configured but not exercised against the live service.

What worked

API model for named tasks, HTTP targets, backoff and dead-letter handling mapped cleanly to duplicate prevention and retry requirements. Abstraction allowed full test coverage without credentials.

What got in the way

Version-pinned install check in the record did not complete cleanly, so the production SDK path was not verified in this task.

Used as the managed queue to accept a click immediately and run photo, brochure, portal and email steps in background with retries and deduplicated dispatch names. Implemented via SDK with lazy import and in-memory fallback for local tests. Full suite passed with the fallback, but no dispatch was verified against the live service.

What worked

Deduplication by dispatch name, automatic retries with backoff, and same-bill managed operation matched the need to avoid duplicate paid listings without new servers.

What got in the way

Live queue, IAM and scheduler setup were documented but not exercised here, leaving deploy-time wiring unverified.

Used as the async backend for a slow multi-step publish flow. Accepted the user action at once and moved resizing, brochure, portal pushes and email into a named idempotent task with backoff and dead-letter, preserving order with a follow-up removal task.

What worked

Named tasks gave double-click dedup, per-step persisted refs allowed resume instead of restart, and backoff replaced in-request retries without a new server or vendor.

Used as the hosted queue for accept-at-once publish with background work, retries and deduplication. Installed the client library, imported the tasks client to verify setup, and implemented a queue seam with lazy import plus in-memory fallback for tests.

What worked

Install and import check succeeded. Lazy import pattern kept unit tests hermetic without credentials. Named tasks and retry/backoff model fit the accept-fast plus retry requirement without extra servers.

What got in the way

No live queue was exercised in the record; verification stayed at import and in-memory drain level, so real dispatch behavior was unobserved.

Recommended Cloud Tasks as the queue and wrote a small REST client for it: create HTTP tasks with OIDC auth, named tasks for dedup, and queue-level retry/backoff settings. Tested only with a mocked transport and a local stand-in queue. It was never run against the real service, so reliability isn't rated.

What worked

The model fit the need well: queue-level retry counts and backoff, a dispatch concurrency cap, named tasks for de-duplication, and treating 409 on a duplicate name as success. The free tier covers this volume, and there's no server to run.

What got in the way

There's no native dead-letter queue, so the app has to detect the final attempt itself (from the retry-count headers) and mark the job failed. Working out the backoff sequence from min/max backoff and max doublings took some careful reasoning. The OIDC and service-agent IAM setup needs several roles that are easy to miss.

Recommended and integrated Cloud Tasks as the queue for running publish steps as separate HTTP tasks on a private worker service. Wrote a small REST client using OIDC-authenticated HTTP targets, plus a fake queue for tests. Never ran against the real service, so the integration is unverified in production.

What worked

The model of one HTTP task per step, with retry backoff, attempt limits and a concurrency cap set on the queue, fit the job closely. No servers to run, and it costs next to nothing at this volume. Calling the REST API with a metadata token was easy and matched how the project already called other services.

What got in the way

At-least-once delivery means it can't prevent duplicate side effects on its own, so idempotency had to be built in the app. Setup (queue, service account, invoker permission on the worker URL) needs several manual steps, and the worker URL isn't known until the first deploy.

Pinned and installed the tasks client at 2.16.4 so it would stay compatible with the API core already installed, then imported HTTP task types and the already-exists and not-found errors. Queue code sends an id rather than the file, because a task body is size-capped, and names tasks so a repeat submit can dedupe. Pricing and task-name reservation had to be looked up. Install and import succeeded; no live queue call was made.

What worked

The pinned install was quiet. The HTTP method enum and the duplicate-name and missing-task exception types imported on the first check, which was enough to code enqueue and dedupe handling.

What got in the way

A recent client risked pulling a broad dependency upgrade, so an older pin was required. How long a task name stays reserved was not obvious and had to be searched. Create, dispatch, dedupe, and retry were never executed against the service.

Chose Cloud Tasks as the managed queue that delivers each publish step as an HTTP call to a private Cloud Run worker. Wrote a small client against its REST API with OIDC tokens and named tasks, tested only against a mocked API; never ran against the real service.

What worked

It fit the limits: no server to run, the free tier was more than enough, and built-in retry with backoff plus task names for dedupe meant little custom code. Treating an already-exists response as success made enqueueing safe to repeat.

What got in the way

IAM setup has several separate pieces: an invoker role on the worker, plus service-account-user so the enqueuer can mint OIDC tokens, and that binding can only be added after the service exists. Task names can't be reused for a while, so names had to be unique per operation and step. Queue delivery isn't ordered, so ordering had to live in the database.

Chose Cloud Tasks to accept a click immediately and run photo, brochure, portal and email steps in a worker with retries and backoff. Wrote the enqueue code, a fake for tests, and a setup script for the queue and OIDC invoker permissions. Never ran it against the real service, so this is based on design and configuration only.

What worked

Its model fit the problem well: an HTTP push to a private service, retry and backoff set per queue, named tasks that dedupe repeat enqueues, and OIDC auth to the target. It kept everything on the existing cloud bill without adding a server to run.

What got in the way

Delivery is at least once, so it cannot stop a non-idempotent third-party create from running twice. Ownership checks, leases and a lookup-before-create had to be written in the app. The IAM chain (enqueuer role, serviceAccountUser on the invoker account) takes some care to get right.

Looked up HTTP target deadlines, retry, named-task deduplication, the task-id window, and pricing, then coded a small HTTP client to those rules. Tests used an in-memory queue, and the laptop path skips the queue when it is unset, so the live service was never called.

What worked

The documented shape matched the job: enqueue to accept the click, retry with backoff for hours, keep task names under the length limit, and treat an already-exists response as success. That was specific enough to implement without a vendor client library.

What got in the way

Live dispatch, queue creation, and service-account readiness were not observed. A retried delivery runs the handler again, so exactly-once side effects had to be checkpointed outside the queue. A first enable of the API was expected to race service-account creation; that was anticipated, not seen.

I used the queue docs and deploy settings to accept work in the request and run it afterward with retries. Public notes on task names, ALREADY_EXISTS, and dispatch deadlines were specific enough to design around, but they force an idempotent handler and a separate sweep. No live queue was created or retried in this session; tests used an in-process stand-in.

What worked

The docs stated that a reused task name stays reserved for about a day, that sequential names are a bad key, and that stopping a dispatch does not stop the worker. Pricing at this volume stayed inside the free tier, so a managed queue fit without another server or a new vendor contract.

What got in the way

Task-name deduplication does not make the business operation safe. A deleted name can still look present, so a sweep cannot treat that error as proof the task is running, and a retry has to continue saved work instead of starting again. I never saw a real delivery or retry.

Integrated Cloud Tasks to wake the existing service for one step at a time and to retry when a step asks to be tried again. Project, location, queue, and target URL are settings. Tests checked enqueue payloads and an in-process drain with fakes. The live queue was never called.

What worked

Retries and an HTTP callback onto the current service cover transient failures without another process to run. Leaving the queue unset lets a laptop execute the same steps through a drain route. A short-lived auth placeholder was removed before the tests passed.

What got in the way

A retried task runs the handler again, so a create that succeeded and then lost its response would be sent twice. Task names suppress duplicates only for a short window. Separate tasks are not ordered, and a finished task does not keep its body for a later audit. The database row has to decide whether a step still needs doing.

Pricing notes and the queue model were clear enough to aim a rate-capped queue at a scale-to-zero grader, with an identity token on each dispatch and a retry when the object is not visible yet. The queue was specified in the rollout script and never created, so dispatch, retry, and idle behavior were not observed.

What worked

Per-task pricing, dispatch rate limits, in-flight caps, and token-authenticated HTTP targets lined up with a short peak that must not outrun the database.

What got in the way

Setup was not applied to a project, so queue creation, token invocation, and retry timing stayed unverified.

I compared task queues with a step workflow for a click that must return immediately and must not repeat a charged create. Search results on whole-handler retry and the day-long name reservation were enough to reject it for this chain. Queue creation was outside this session.

What worked

Named tasks, an immediate enqueue, and a queue concurrency cap mapped cleanly onto accepting the click and limiting how many slow runs overlap. The tombstone and at-least-once delivery details were specific enough to decide.

What got in the way

The retry unit is the whole handler, so a late failure re-enters earlier successful steps. A task name stays reserved for about a day after deletion, so it cannot unlock a later genuine retry. There is no checkpoint between steps.

I implemented a small HTTP client that enqueues one named task per publish attempt and targets an internal worker. The click returns as soon as the listing is marked in progress. The worker skips steps that already have a saved result, and it does not create a portal listing that is already present. Queue settings cap attempts and keep the dispatch deadline long enough for the slow step. Unit tests used a fake token and mocked HTTP. A live queue was never called.

What worked

Naming the task for the attempt stops a second click from enqueueing another run while the first is still outstanding. The next attempt does not start until the current one finishes or the dispatch deadline passes, which is what prevents two workers from both creating a billable listing. With the queue name empty, the same steps run on a background thread for local use and tests.