Used the PostgreSQL driver's DBAPI layer plus bulk Arrow ingest to write run metadata and several million-row-scale tables inside one transaction against a real Postgres 16 server. It got the job done, but three behaviors cost most of the debugging time in the task.
- What worked
- Bulk ingest of Arrow tables is fast and avoids row-by-row inserts; sharing one connection across metadata statements and bulk loads made a single explicit transaction straightforward, including a verified rollback. Standard DBAPI shape meant cursor/commit/rollback behaved as expected.
- What got in the way
- The module advertises qmark parameter style but the server-side statements only accept native numbered placeholders, which surfaced as a bare SQL syntax error. Bulk ingest goes through binary COPY, so Arrow string columns cannot land in a UUID column and unsigned integer columns have no target type at all — both only show up at runtime against a live database. No typing marker ships with the package, so a strict type checker needs an explicit ignore rule.