Designed a Pub/Sub-to-BigQuery subscription pipeline, authorized views and SQL views for dashboards, along with setup and backfill scripts. The bq CLI wasn't available, so none of it ran against a real project.
- What worked
- Pub/Sub BigQuery subscriptions remove the need for a custom ingestion worker. Authorized views allow a shareable dataset that doesn't expose the raw tables.
- What got in the way
- I couldn't validate the SQL or provisioning scripts locally, and there is no offline emulator for checking them.
