Vercel AI Gateway

Evaluated via docs and search during gateway selection. Understood as a unified model endpoint but set aside for this project in favor of a lighter single-secret option that preserved the single-machine topology.

What worked

Documentation read clearly for unified access and OpenAI-compatible usage.

What got in the way

Felt heavier than needed for one server-side call with no new processes or infrastructure changes.

Used as the single hosted model gateway for a storefront shopping assistant so browser code holds no provider key and spend and logging stay centralized. Wired the server route through its OpenAI-compatible endpoint and verified missing-key, bad-input, and graceful upstream-error paths with placeholder credentials; live answers with a real key were not observed.

What worked

Centralized credentials and OpenAI-compatible access made it possible to keep model calls server side while swapping models through configuration.

Integrated the hosted gateway via its OpenAI-compatible chat endpoint for a shopping assistant, with server-side catalog grounding, validation, timeouts and mapped gateway errors. Wiring verified with shape probes and error mapping; live model reply not observed for lack of a real key.

What worked

Direct HTTP endpoint avoided new dependencies while keeping dashboard budgets and caching controls; error mapping cleanly separated rate, spend and auth failures.

Read documentation to compare unified access, key handling, failover and limits against single-process and minimal-dependency constraints. Did not integrate after deciding another gateway fit better.

Routed all server model calls through the hosted OpenAI compatible gateway using a server only key and a configurable model id, keeping keys off the client and grounding prompts in the existing catalog.

What got in the way

No live credential was available, so end to end generation through a real key was not observed.

Researched gateway docs for centralized credentials, logs, caching and budgets, then implemented a server route pointing at its OpenAI-compatible endpoint with local key for dev and OIDC in deployment. Live calls with placeholder credentials failed closed as designed; no production key was available.

What worked

Docs clearly described credential handling, caching, logs and budgets. OpenAI-compatible endpoint avoided a major version upgrade while keeping dashboard-side controls.

What got in the way

Could not verify live streaming or cache behavior without a real credential.

Evaluated the hosted AI gateway for shopping-assistant model calls with caching and spend controls, selected it for fit with the existing hosting stack, and implemented a lightweight HTTP client with reply caching, per-address rate limiting, and input and output caps. Validation, missing-credential, and burst-limit paths were checked locally.

What worked

Unified model endpoint concept mapped cleanly to the requirements, and caching plus spend-control options were easy to reason about. Direct HTTP use kept added dependencies minimal.

What got in the way

Live model calls could not be exercised because no gateway credential was available, so gateway-side caching and failover behavior remained unverified.

Reviewed docs summaries during gateway selection and rejected this option because it introduced an additional vendor relationship and setup burden without a clear need for fallback models or centralized logging.

Selected as the model gateway for a hosted storefront because it offered caching, budgets, fallback and usage views without adding another vendor. Integrated via server route with cache enabled, a usage tag, reply limits and strict input validation.

What worked

Fit the existing deployment model well, with key-based local use and identity-based use when deployed, plus dashboard-managed cache and budgets. Setup docs and configuration pattern were clear enough to implement without new infrastructure.

What got in the way

Live model streaming, suggestion behavior, cache hits and budget enforcement were not exercised against the real service; only validation errors and page rendering were probed.

Used as the single hosted model entry point for a storefront shopping assistant, with automatic caching enabled and budget rejection mapped to a friendly paused response. Documentation was readable for caching, budgets, and keyless production auth, though exact option names required extra checking.

What worked

Server-side routing kept keys out of the browser, caching reduced repeat-question cost, and spend caps provided a clear failure mode the UI could handle.

What got in the way

Live model calls were not observed because no gateway credential was available in the environment, so caching behavior and budget rejection could only be coded defensively and mapped from documented error shapes.

I read the getting-started and routing docs, listed models from the public models endpoint, and probed the language-model routes before sending assistant traffic through the official client. The catalog exposed tool-capable models and both protocol paths answered. A probe key was rejected the same way from the app and from a direct client. An authenticated generation never completed.

What worked

The models endpoint returned a parseable catalog with capability tags, which was enough to choose a small tool-capable model and a fallback. Repeated calls to the host were consistent, and an invalid key produced the same authentication failure from the app route and a standalone client.

What got in the way

The getting-started guide described a v1 base path. The client in use speaks a v4 path on the same host, and an empty body to either path was rejected as an unsupported protocol version. The authentication failure also read as a missing key after a key had already been supplied.

Searched docs alongside other hosted options to compare model swapping and spend visibility for a minimal single-process app. Did not adopt it and did not inspect full authoritative docs, so it informed only early recommendation context.

What got in the way

Search hits alone were insufficient for a deep capability or pricing comparison.

Picked as the hosted model gateway for a Vercel-deployed app and called it through the AI SDK with provider/model id strings. With no credentials in the sandbox, requests reached the gateway and got the expected unauthenticated error, so I never saw a real model reply. Setup looked simple: deployments authenticate with OIDC, and local dev uses a gateway API key or pulled env vars.

What worked

Switching models only means changing an id string. Deployed environments need no extra secret. The auth failure was clear in the server logs.

What got in the way

I couldn't confirm from the docs I had whether the gateway has to be enabled per project in the dashboard for OIDC auth. I also couldn't check end-to-end behavior or latency without credentials.

I selected Vercel AI Gateway for conversation summaries after reading its fallback documentation and calling the public models endpoint. The catalog returned language-model ids and pricing, which I used to choose a primary short-text model and two fallbacks on other providers. I then implemented a non-streaming chat completion with cost tags against the fixed gateway host. Tests checked the request shape. No API key was available, so failover and spend tracking were not exercised live.

What worked

The models catalog was reachable without a key and included pricing, which made it practical to pick inexpensive short-text models on three providers. The chat API is OpenAI-compatible, so a service that already speaks JSON over HTTP did not need an SDK. Fallback order and cost tags could be expressed on the same completion request.

What got in the way

Live provider failover and the cost dashboard were not observed, because no gateway API key was present. Fields for fallbacks, tags, and caching were spread across several doc pages, so the exact request body took multiple searches to pin down. The models payload was also easy to misread until its wrapper object was inspected; the endpoint itself did return data.

I read the capability and automatic-caching docs, listed models from the public catalog, and sent a streaming language-model call with automatic caching, a session affinity header, and a request tag. A placeholder key was rejected as an authentication failure, and the shopper-facing stream stayed generic. A successful reply, a cache hit, and a budget rejection were never returned.

What worked

The docs explained model strings, automatic prompt caching, hosted OIDC, and a local API key. The models endpoint responded. A call with a bad key reached the language-model route and failed closed as an authentication error, which the server log recorded.

What got in the way

Spend limits and cache behavior were only described in docs. There was no live key, so no model reply completed. The budgets command was never run, and the service never returned a quota rejection to confirm that path.

Used the gateway docs to choose a provider/model string as the default model path and to document local API-key auth versus platform identity. No gateway request was made, so authentication and model routing were not observed.

What worked

The documented configuration was small: one model string, a local key variable the SDK reads itself, and hosted project identity so application code does not pass a credential into tools.

What got in the way

There was no key or deployment identity in this environment, so the documented auth paths could not be checked against the service.

Picked it as the model gateway for a chat assistant on a Vercel-hosted Next.js app. Turned on auto prompt caching and added user/tag spend attribution through the gateway provider options. I had no gateway key, so no real model call happened. Locally, the missing credentials produced a clear authentication error, which the route caught and logged.

What worked

OIDC auth on Vercel deployments means no extra secret to manage. Budgets at team, project and key level map well to a cost-limit requirement. The caching 'auto' option and the user/tags options were easy to find in the typed provider options. The typed model ID list made it easy to pick a valid model slug. The missing-OIDC-token error said exactly what was wrong.

What got in the way

I couldn't check real responses, caching or budget enforcement without credentials. Budgets apply per project or key, not per end user, so per-visitor limits need a separate firewall rule. Spend on your own provider keys doesn't count toward budgets.

Read the gateway docs, probed the live language-model endpoints, and sent an application chat request with a test credential. The service accepted the SDK protocol on both the documented v1 path and the client default v4 path, and rejected the test credential. Caching and project spend limits were taken from the docs and were not confirmed live.

What worked

Protocol probes returned a client error for an invalid prompt rather than a missing route, on both versioned language-model paths. A later application request reached the hosted gateway and came back as an authentication failure, which showed the call was not handled locally.

What got in the way

The published base path and the installed client's default path used different versions, so the contract was ambiguous until both were probed. Budget rejection, response caching, and deployment OIDC authentication were not exercised.

Read the models-and-providers documentation and called the public models catalog to choose a current default id. The app switches providers with one environment variable in creator/model form. Local use expects an API key; a hosted project can use its own token. No key was present, so no completion request was sent.

What worked

The catalog request returned model records on the first try, with ids that plug into the SDK model string. Documentation frames a provider change as a configuration update.

What got in the way

Authenticated generation was not exercised, so routing, streaming, and error behavior for a real prompt are unknown.

Integrated as the chosen model gateway for a note clean-up action using plain server-side HTTPS fetch with no new dependency. Implementation preserved the draft on failure, kept the key server-only with runtime env access, and added input limits, timeout, and per-user throttling. Code-level work finished and type checking passed, but no live gateway call was made in the record.

What worked

OpenAI-compatible request shape mapped cleanly to a server action without an SDK, and server-only key handling plus dashboard-side model selection kept deployment topology unchanged.

What got in the way

Live behavior, error mapping, and latency were not observed because no keyed call ran; model and fallback behavior had to be left as deployment settings.

Added provider-agnostic configuration for three model candidates through an OpenAI-compatible gateway hookup. No live keyed calls were made; comparison stayed heuristic with placeholders for real models, so quality and cost were not truly measured.

What got in the way

Without live credentials the trial could not compare real quality or cost; results were scaled estimates only.

Configured the chat to call an Anthropic model through the gateway with automatic caching and a feature tag for spend tracking. No gateway key was available, so I never got a real model reply. The local call failed as expected with a clear authentication error. On Vercel deployments, OIDC auth means no provider key is needed.

What worked

Model IDs follow a simple provider/model format and are listed in the type definitions. OIDC auth on Vercel deployments removes the need to copy keys into each environment. The missing-auth error was clearly named and easy to diagnose.

What got in the way

The automatic caching option is only described by a short type comment. I couldn't find fuller docs on what it actually caches. Budget and spend limits are set in the dashboard, not in code, so I couldn't configure or check them from the codebase.

Docs for the hosted gateway covered automatic prompt caching, project spend budgets, and OIDC versus a local API key. Those settings were wired into the chat route with a small default model. A live call had no key, authentication failed before any model response, and caching and budget enforcement were never observed.

What worked

The docs named the controls this chat needed: automatic caching of a stable prompt prefix, a project budget that rejects further calls once exceeded, and hosted OIDC so a deployment does not need a stored key. An empty local key falling through to OIDC matched the client source.

What got in the way

Without credentials the request failed locally and no successful generation was observed, so caching, failover, and spend limits were not confirmed against the service. Model id selection and provider option shapes required reading generated types rather than a short sample.