Used as the maintained eval runner for transform and report prompt surfaces with fixture cases and deterministic assertions. Validation and local enumeration worked; live model runs were blocked by the missing API key in the environment. Pinning to an older minor avoided a dependency conflict.
- What worked
- Config validation and local eval enumeration worked once provider shape and prompt entries were corrected. File-based cases with scripted providers fit the existing stack without new infrastructure.
- What got in the way
- Current docs did not match the installed version's custom provider shape, requiring inspection of shipped code to find the expected class form.
