Used as file-based eval runner for two versioned suites with deterministic checks and a baseline comparison script. Resolved a major version conflict by pinning an older release, then validated grading with canned outputs and end-to-end runs without a model key.
- What worked
- Local execution kept customer cases in-repo, threshold and assertion behavior was testable with canned outputs, and CI gating via exit code was straightforward.
- What got in the way
- Latest release conflicted with the existing model library peer range, and config keys for thresholds and external assertions were hard to confirm from installed types alone.
