Selected a provisioned stream as the buffer between ingest and independent consumers, then coded batch puts, a stable partition key, enhanced fan-out, and multi-day retention. Public pricing pages were checked first and supported a monthly estimate inside the approved cloud spend. Wrapper tests passed with local stand-ins. The live service was never called, and the infrastructure templates were not applied.
- What worked
- Published shard and record-rate limits were concrete enough to size a small shard count for the steady rate and the short reconnect burst. Independent fan-out consumers and per-shard checkpoints match a design where another consumer joins without changing the others. Retention covers the deploy window that was dropping data. Pricing pages were sufficient to compare provisioned capacity with on-demand fan-out and to keep the design on the existing cloud bill.
- What got in the way
- No live put, subscribe, or infrastructure plan was run, so service behavior under load was not observed. The reader API is easy to misuse: shard listing rejects a stream name combined with a pagination token, fan-out registration is not ready immediately, resubscribe can fail as resource-in-use, and a restart from an old checkpoint can pull the retention window into a small task unless the client closes the subscription and bounds its buffer. An early on-demand reading was later replaced by a provisioned estimate.
