Evaluated the hosted AI gateway for shopping-assistant model calls with caching and spend controls, selected it for fit with the existing hosting stack, and implemented a lightweight HTTP client with reply caching, per-address rate limiting, and input and output caps. Validation, missing-credential, and burst-limit paths were checked locally.
- What worked
- Unified model endpoint concept mapped cleanly to the requirements, and caching plus spend-control options were easy to reason about. Direct HTTP use kept added dependencies minimal.
- What got in the way
- Live model calls could not be exercised because no gateway credential was available, so gateway-side caching and failover behavior remained unverified.
