The TypeScript SDK and CLI supported datasets, experiment runs, seeded sampling, custom scoring, and regression reporting. Type declarations were detailed enough to implement the suite, but local discovery required substantial inspection and even listing evaluations required credentials, so no live service run was observed.
- What worked
- Its APIs covered the full desired workflow: versioned test data, immutable evaluations, deterministic sampling, custom reporters, and CI gating. The SDK integrated with the application's existing TypeScript functions, avoiding duplicated prompt definitions.
- What got in the way
- The CLI refused a local evaluation-listing command without an API key. SDK type expectations also exposed mutable-versus-readonly tag incompatibilities that required code changes. Live behavior could not be verified without service credentials.
