Reviewed hosted evals through search and the installed SDK to see if they could catch report-builder regressions. The product models prompt-to-output, not generate-then-execute, and the hosted platform is being retired, so it was not adopted.
- What worked
- It was easy to discover that evals appear in the existing SDK and that the hosted offering is going read-only then shutting down, which made the reject decision clear.
- What got in the way
- Hosted evals would not score the in-process transform step that produces wrong totals, and a service with a near-term shutdown is a poor fit for a small team that needs something they can keep running.
