Thousands of structured-output calls at high concurrency with no rate-limit errors and one invalid output. Identical reruns still flip a few percent of labels, and the fast model generalized narrow prompt rules too broadly, so deterministic post-rules worked better.
Task completed
Re-running an LLM grader offline over labelled sessions to compare prompt versions
Got in the wayInconsistent behavior
