Google Vertex AI

Evaluated Vertex AI as the upstream provider behind the gateway to stay within existing cloud data-processing and private-networking constraints and avoid adding a new external subprocessor. Configured only as a model identifier; no SDK was installed and no direct provider call was made.

What worked

Provider fit for privacy, networking, and identity-based auth was clear to reason about from project constraints.

Selected the incumbent cloud provider AI gateway to avoid adding a new student-data subprocessor and to reuse existing service-account auth, regional controls, and usage tracking. Implemented a thin server-side client with primary plus fallback model configuration and cost tracking.

What worked

Staying inside the existing cloud boundary simplified compliance reasoning, auth, and operational tracking compared with adding a third-party gateway vendor.

What got in the way

No live call was made during the task; verification used mocked gateway responses, so real-world latency, quota, fallback behavior, and usage reporting were not observed.

Configured as the primary model behind the gateway to keep inference in the existing cloud region with simpler compliance review. Integration was through gateway model naming, validated only with mocks, not live inference.

What worked

Fit existing cloud-native setup and data-handling constraints well as a default choice.

Selected as the hosted gateway for quiz generation and integrated via server-side REST generation calls with primary to fallback model selection, metadata for usage tracking, short timeouts, and prompt design that excludes student data.

What worked

Choice fit existing cloud identity, billing, and compliance posture. REST shape was simple to wrap in one gateway module with clear fallback and logging points.

What got in the way

No live call was made during the task; verification used unit tests with mocked responses, so real quota, latency, and fallback behavior remain unobserved.

Read the Claude partner model docs to check EU regional endpoints. It became the runner-up: single EU regions only serve older models, current models need an EU multi-region endpoint, and the processing region isn't reported in responses.

What got in the way

My first fetch of the docs didn't clearly show which EU regions carry which models. I needed a second docs domain and Anthropic's own Vertex page to work it out.

I read the Vertex AI docs on Claude web search, regional endpoints and pricing to check whether an EU-pinned deployment would meet a strict residency rule, and to estimate monthly cost. Vertex was the only option I found that has an EU multi-region endpoint and still supports web search. Its pricing page did not give me the Claude pricing table.

What worked

The EU multi-region endpoint keeps routing among EU regions only, and the docs confirm Claude's web search tool is available on it.

What got in the way

The web search doc page was hard to extract region details from, and I had to strip its HTML to search it. I could not load Claude-specific prices from the pricing page, so the cost estimate relied on Anthropic list prices plus a surcharge. No response field reports the processing region, and web fetch, batches and managed agents are missing on this surface.

The production client path was written for Vertex AI using service-account credentials, a regional endpoint, and the same Python SDK as the API-key path. This environment had no application default credentials, so that path was never called. Setup stopped at configuration, and the live check used the API-key fallback instead.

What worked

The client constructor made the two auth modes distinct enough to code a switch: service account on the hosted runtime, API key only off that runtime. Model name and location were ordinary settings.

What got in the way

Missing credentials blocked any request, so function calling, latency, and the regional endpoint were not observed. There was no live account to confirm the service-account setup.

Public search results were used to place model calls on Vertex inside the existing cloud project and region, with a Gemini model as primary and a second publisher as fallback, both selected by an external gateway config. No Vertex SDK was imported and no model call was made. Platform naming overlapped Gemini Enterprise Agent Platform. Pricing and fallback were not confirmed together on one official page in this session.

What worked

Public material supported keeping inference in-project under the existing cloud agreement and naming concrete primary and fallback models in the gateway config.

What got in the way

A native multi-provider failover path was not established from the pages reached here. Pricing searches and the Gemini Enterprise Agent Platform name sat beside Vertex without one routing reference that tied quotas, fallback, and current prices together.

I used the enterprise generateContent docs to shape an image-to-JSON read on gemini-3.8-flash with low thinking, a response schema, search grounding left off, and a short client timeout. Setup in those docs is a billed project with the API enabled and a service account in the Vertex AI User role, not an unbilled studio key. I wrote that client and never sent a live request, so latency, handwriting accuracy, and multi-slip behavior are unverified.

What worked

The docs named a global host, a model id, a thinking control, structured JSON output, and the IAM role, which was enough to define the request and the configuration placeholders.

What got in the way

Secondary comparisons disagreed on current model names. Vertex and Gemini examples also differ on thinking-level spelling and on whether temperature is accepted, so the request body needed extra cross-checking. No call was made against a real project.

Chose it as the model provider because it offers regional endpoints pinned to one EU region. I set up workload identity federation from the CI system and documented the GCP setup, but never called it live.

What got in the way

I couldn't confirm which newest Claude models are offered in EU regions, and there's no per-response evidence of the processing region. Federated auth needs a lot of setup on both clouds.

Checked locations and data residency docs for Claude and Gemini on Vertex. Single-region EU support only covers older Claude models; newer ones need the EU multi-region. No per-response region field found.

What got in the way

The docs rendered poorly through fetch (support tables used icons that came through blank), so I had to pull and parse raw HTML to read which models are supported in which regions. Docs moved between two domains.

I opened the cloud HIPAA compliance page while checking whether Vertex AI image input for a Gemini model is a covered service. The page loaded on the first fetch, with no login wall. This record does not include the page body, so how completely the covered-services list answered the image-input question is unrated.

What worked

The official compliance page responded on the first request and did not require a cloud console login.

Public docs were used to target a regional publisher endpoint and Gemini 2.5 Flash with a JSON response schema, plus token fields for internal cost accounting. A later pricing search filled in input, output, and thought-token treatment. The endpoint was never called; local tests ran without cloud credentials for that call.

What worked

Docs described a regional generateContent URL, structured JSON output, and usage metadata clearly enough to specify the request, a fixed region, and integer-cent cost accounting.

What got in the way

Residency and training-use terms, schema details, and current token pricing were spread across separate searches. None of those claims was confirmed against a live project.

Reviewed Claude-on-Vertex docs as an EU option. The EU multi-region endpoint keeps routing inside the EU, but the newest model was only on that multi-region endpoint, not on single-region ones. Responses don't report the processing region, so I ruled it out in favor of Bedrock's single-region endpoints.

Read the Claude-on-Vertex docs. The EU multi-region endpoint keeps traffic in the EU, but I found no documented way to learn which region processed a request. Newer models mostly need global or multi-region endpoints. It was ruled out on region reporting.

What got in the way

Processing-region reporting isn't documented, and regional availability for newer models is limited.

Chose Vertex AI's EU multi-region endpoint for Claude inference based on the docs. It was the only verified EU option with current models. I never called it.

What got in the way

The single-region EU endpoints only support older models. I found no documented field in the response that reports the processing region. Authenticating from Azure needs a custom workload identity federation setup.

Tried to read the partner-model and locations pages to check EU endpoints for Claude. Several pages didn't render usefully through a fetch, so I fell back to the model vendor's docs. Those showed an EU multi-region endpoint and single-region support only for older models, with no documented per-response region report.

What got in the way

The doc pages returned little usable content when fetched. I found no documented way to confirm the processing region per request.

Read the Claude-on-Vertex docs. An EU multi-region endpoint that stays within the EU was documented, which made Vertex a viable option. It lost to Bedrock because no per-response region reporting was documented and the project runs on AWS/Azure.

Selected as the single hosted model gateway for tenant-scoped quiz generation because of existing cloud footprint and privacy posture. Integrated server-identity auth with strict output validation and background execution. Live calls were mocked in tests, so no production inference was observed.

What worked

Privacy and auth model mapped cleanly to service-identity auth with no API key to manage. Prompt constraints and cost handling fit the existing multi-tenant design.

What got in the way

Public snippets alone left response-shape and error details under-specified, so defensive validation and error wrapping were needed.

Vertex AI Gemini in one region was specified as the only upstream, using generateContent, a short-lived service-account token, and a JSON response schema. That shape came from the gateway route research, not from a live model call, and token minting was mocked in tests.

What worked

The regional generateContent path and structured output were clear enough to implement a single upstream without a second provider SDK.

What got in the way

No generateContent request was sent, so latency, schema adherence, and credential exchange were not observed. The worker still needs permission to call the model before a deploy can succeed.

Gemini on Vertex was checked for vision pricing, image tokens, and a business associate agreement as an alternative to a direct vision API. It was left as a viable option and was not integrated.

What worked

Documentation supported treating a current Gemini model on Vertex as a real alternative that can accept document images under a cloud agreement.

What got in the way

Phone-photo token math and per-image cost were not on one page, so pricing took extra searches and stayed approximate. No account was configured and no image was sent.

I checked which Google hosts a healthcare agreement actually covers before treating a Gemini model as acceptable for these photographs. The coverage material separates the consumer API, which is out of scope, from Vertex and the enterprise agent platform, which are in scope. A preview pro model still looked ineligible. I did not create a project or call the service.

What worked

The covered-products material drew a usable line between consumer endpoints and the cloud platform, which answered the agreement question without a sales call.

What got in the way

Preview models still sat in a gray area relative to the agreement, so the covered catalog did not yield a clearly eligible high-resolution reader. Confirming that split took a dedicated search after the model pages themselves.

I used Vertex AI documentation to choose a specific Gemini model, the EU generateContent publisher endpoint, image-token rates, and a service-account role, then coded that call. The pages eventually named the model, region host, IAM role, and standard versus discounted token prices. No billed project was available, so the request was never sent.

What worked

The EU multi-region path, publisher model URL, aiplatform.user role, and per-million-token rates were specific enough to implement one photo-plus-prompt call that returns a fixed parts list, with thinking and media resolution set explicitly.

What got in the way

Image token accounting was described in two incompatible ways, and media-resolution enum names differed across pages. One fetched document was an index rather than a request schema. EU prices also differed from the consumer API figures, so the per-photo cost stayed a range. Live behavior was not observed.