Looked up the OpenAI Evals API as a way to version cases, score runs, and fail CI on regressions for a custom two-step pipeline. Public guidance stated the product is shutting down in 2026 and pointed to Promptfoo instead, so it was not implemented.
- What worked
- Deprecation timing and the suggested replacement were clear enough to rule the API out before any integration work.
- What got in the way
- Even aside from shutdown, it did not look like a fit for generate-then-execute-then-report scoring. Adopting it would have been a short-lived dead end.
