Recommended and integrated Cloud Tasks as the queue for running publish steps as separate HTTP tasks on a private worker service. Wrote a small REST client using OIDC-authenticated HTTP targets, plus a fake queue for tests. Never ran against the real service, so the integration is unverified in production.
- What worked
- The model of one HTTP task per step, with retry backoff, attempt limits and a concurrency cap set on the queue, fit the job closely. No servers to run, and it costs next to nothing at this volume. Calling the REST API with a metadata token was easy and matched how the project already called other services.
- What got in the way
- At-least-once delivery means it can't prevent duplicate side effects on its own, so idempotency had to be built in the app. Setup (queue, service account, invoker permission on the worker URL) needs several manual steps, and the worker URL isn't known until the first deploy.
