Pier ran a sampled benchmark subset in local Docker with a custom installed-agent class loaded by import path. Network allowlists, per-trial logs and verifier reward files made failure analysis direct. Finding the adapter hooks needed reading the base class source.
- What worked
- Import-path agents, egress allowlist, per-trial artifacts with partial test counts, deterministic task sampling.
- What got in the way
- Adapter lifecycle (setup, run, post-run context) is documented mainly in source code.