Read the CI evaluation docs while comparing hosted eval products to an in-repo library. The pages made versioned datasets, experiments, git-aware baselines, and GitHub Actions regression blocking easy to map onto the checklist. The extra SaaS and API key were a poor fit for a team that wanted no new hosted product, so it was not installed.
- What worked
- The CI docs were specific enough to judge baseline comparison and fail-on-regression without a trial account.
- What got in the way
- Adopting it would have meant a second hosted service and key on top of the existing model vendor, which the operational constraints ruled out.
