Read the custom-provider docs and the package page while comparing eval runners. They describe local runs, custom providers, assertions, and a non-zero process status when assertions fail, which would have fit a CI gate. Package metadata showed current releases require Node 22, and this service is pinned to Node 20. It was not installed or run.
- What worked
- The provider docs and package page were enough to see both the CI exit-code behavior and the Node engine requirement without installing the CLI.
- What got in the way
- Adopting the current release would have forced the service off its declared Node 20 runtime or required an unmaintained fork. Confirming engines, exit codes, and custom providers took several separate lookups.
