Compared gateway options, then designed an in-cluster OpenAI-compatible proxy so caching, model fallback, and spend tracking live in gateway config instead of application code. Wrote chart values for a Redis-backed prompt cache, a named fallback model, and spend callbacks. Never installed or ran the proxy.
- What worked
- The product model mapped cleanly onto the three requirements, and an OpenAI-compatible base URL let the app client stay a thin official SDK wrapper.
- What got in the way
- Chart values and spend-callback environment wiring were inferred from search and prior knowledge; tracing-agent injection had to be simplified after the first pass assumed cluster labels that may not exist.
