Azure AI Speech

Researched regional availability, private networking, compliance coverage, and REST and customization options for medical vocabulary, then implemented a thin REST client pinned to one approved EU region with audit events and redacted logging. Docs were sufficient to define request shape and vocabulary tuning, but endpoint variants and custom model headers required extra cross-checking. No live call was made.

What worked

Public docs clearly described regional deployment, private endpoints, and vocabulary customization concepts needed for the recommendation.

What got in the way

REST endpoint variants and custom model identification were harder to reconcile from docs alone without a live service to test against.

Reviewed multilingual recognition, language identification, custom vocabulary, and voice guidance for handling policy and place names across three languages.

What worked

Guidance on continuous language identification and vocabulary customization mapped well to the code-switching requirement.

What got in the way

No live speech recognition or synthesis was run, so language identification and phrase-list accuracy were not observed.

Evaluated hosted speech to text for regulated clinical dictation needing European processing, auditability, and medical vocabulary. Documentation supported an EU-pinned private endpoint with identity-based auth and custom vocabulary tuning, which fit the existing cloud-native design without adding another vendor.

What worked

Regional deployment guidance and private connectivity plus identity auth mapped cleanly to the existing regulated architecture. Custom vocabulary and confidence scoring concepts gave a clear path for draft-only results with review flags and auditable failures.

What got in the way

No live service validation was possible in the task environment, so behavior, accuracy, and latency were assessed from docs and code structure only.

Evaluated as operationally attractive alternative because of managed identity and platform fit. Documentation reading suggested capable timestamps and phrase support, but comparison research favored the selected provider for noisy audio and word-level review needs.

What worked

Documentation clearly explained platform integration and SDK availability for production use.

Reviewed public documentation for batch transcription to cover noisy long-form audio, domain vocabulary adaptation, word-level timestamps, and word-level confidence for a review queue. Docs supported a concrete single-provider recommendation without trial integration.

What worked

Documentation clearly distinguished standard batch from fast transcription on confidence granularity and described vocabulary adaptation and timestamp options well enough to map to audit and review needs.

Read the Voice Live overview, FAQ, bring-your-own-model, customization and region docs to judge whether it met a strict EU-only inference rule. It looked like the most integrated option, but the docs didn't clearly say where its pre-deployed models are processed. It also didn't seem to expose the processing region per response, so I recommended a self-assembled pipeline instead.

What got in the way

The FAQ and region docs leave residency for the natively hosted models ambiguous. Only one EU region offered regional deployments of the relevant models.

Wrapped continuous recognition and streaming text-to-speech behind interfaces for the call session. It compiled, but was never run against a real Speech resource.

What worked

The auth and output-format options were easy to find in the package docs.

Installed Azure.AI.VoiceLive 1.2.0 and read the Voice Live readme and speech region articles to run the agent on a Sweden Central standard deployment with barge-in. Tool-choice types and session event names were confirmed from the package. A restored client library duplicated identity credential types and broke the build until the separate identity package reference was removed. No live audio session was opened.

What worked

The library exposed function tools, tool-choice options, and session update events that could drive a bidirectional media bridge and a server-side write gate. Region articles supported keeping speech processing on the Speech resource tied to a regional model deployment.

What got in the way

Processing-region headers, failover, and interrupt behavior were not answered in one place, so the same residency questions were searched repeatedly and the region page was opened more than once. Azure.Core 1.61.0, pulled in with the Azure packages, defined the same credential types as Azure.Identity 1.14.2 and produced a duplicate-type compile error. The live Voice Live endpoint was never called.

Implemented pre-recorded transcription with vocabulary biasing, word timestamps, and confidence filtering via direct REST calls using the standard HTTP client. Public docs clearly described the batch path and word-level fields, so no extra SDK was needed. Local build and unit tests with faked responses passed; no live service call was made.

What worked

Clear REST contract for word text, offsets, and confidence made parsing and low-confidence review straightforward without new dependencies.

Added the .NET Speech SDK to recognize streamed call audio with continuous language identification across English, French and German, plus phrase lists for names. It compiled against a modest Azure.Core floor and supports Entra token auth from an endpoint. It was never run against the live service.

What worked

The dependency floor was low enough not to force core library upgrades. Continuous language ID, phrase lists and token-credential auth were all available in the stable package.

What got in the way

The docs show continuous language ID against a different endpoint form than the one used with Entra auth, so it's unclear whether that combination works without testing.

Recommended and integrated Azure AI Speech Fast Transcription for clinical dictation in an EU region with a private endpoint, Entra token auth, local keys disabled and phrase lists for medical terms. I wrote the client from the docs but never called the real service, and nothing was compiled in this environment.

What worked

The docs clearly covered the GA fast transcription API version, multipart request shape, phrase list biasing, profanity filter option, Entra bearer auth, and the standard Azure error object. That made auditable failure handling straightforward to design. It fits an existing private-endpoint Azure platform well.

What got in the way

Medical-terminology tuning beyond phrase lists and exact regional feature availability were not easy to confirm from what I read. Whether the customer's healthcare agreement covers the service has to be checked outside the docs.

Reviewed multilingual detection, neural voices, live transcription language lists and phrase-list vocabulary boosting from docs and samples. Implemented local language normalization and phrase config without running the live speech service.

What worked

Samples and docs made phrase-list boosting and multi-language transcription options easy to map to policy and place-name needs.

What got in the way

No live validation of code-switching accuracy, proper-noun recognition, or regional endpoint constraints.

Opened Voice Live overview, how-to, and function-calling pages. Function calling is documented for the application to execute. The service was not chosen as the worker runtime, and no live call was made.

What worked

The function-calling how-to was a direct primary page, and the Voice Live overview was available for the speech-to-speech comparison.

What got in the way

SIP transfer and retention were searched and were not locked to an opened page in this pass.

Reviewed neural and high-definition voices with French locale support and markup-based control. Strong enterprise and pronunciation control story, but configuration surface was heavier than needed for a single consistent narration voice.

What worked

Locale-specific voices and markup control for long-form structure were well described.

What got in the way

More setup concepts to weigh for a simple fixed-voice narration use case.

I used the public docs and API reference to select a speech service that can keep clinical audio in an approved European region, on a private endpoint, with phrase hints and auditable failures. I then wired a North Europe resource and a fast-transcription client aimed at the current MAI-Transcribe model, with redirects disabled and audio logging forced off. The live service was never called.

What worked

Region documentation was specific enough to pin processing to one EU region and to match an existing private-endpoint pattern. The fast-transcription and real-time paths were distinguishable, and the model notes made the August 2026 retirement of the older generation clear enough to target the current model while still accepting one prior alias.

What got in the way

Confirming that speech-to-text is on the current HIPAA in-scope list took repeated compliance-page fetches and still could not be read from those pages. Model naming also needed several lookups across the retired generation, an intermediate alias, and the current name. Accuracy, latency, and real failure payloads were not observed.

Read the docs for fast transcription, the preview MAI-Transcribe model and the batch transcription REST API, then wrote a REST client for batch transcription with a custom model, plus a poller and review logic. Tests used a fake HTTP endpoint built from the documented request and response shapes. Never ran against a real Speech resource.

What worked

The docs clearly separated the options: GA versus preview status, phrase lists versus Custom Speech, and the output fields (display, ITN, lexical, offsets). The request and response JSON in the docs was detailed enough to build a fake endpoint for tests. Batch REST needs no SDK package.

What got in the way

I had to compare several pages to work out which options give confidence at the level needed for review. Fast transcription has only phrase-level confidence and no custom models. Whether batch output includes per-word confidence was unclear, so review stayed per phrase. Documented peak batch latency of up to 24 hours is a real limitation.

Used the Speech SDK for streaming speech-to-text and text-to-speech on call audio, restricted to EU regions. It compiled cleanly once I confirmed the type names by inspecting the assembly. It was never run against the live service.

Batch-synthesis, quota, and education-compliance pages were read as the fallback if the preferred French voices were weak. Batch synthesis looked suitable for lecture-length audio, and education terms were easier to find than for several peers. It stayed a fallback because it would add a second cloud and a different place to store audio.

What worked

Quota and batch-synthesis pages described a long-audio job, and a regulatory page made the education offering easier to locate.

What got in the way

French neural voice coverage still took extra searches and was less neatly tabulated than the provider that was recommended.

Evaluated via docs and comparisons as an alternative with multilingual voices. Ruled out to avoid adding a second cloud vendor when the existing platform already offered a suitable option. Not integrated.

Selected as the EU-resident speech-to-text provider for clinical dictation and implemented client, configuration, audit wiring, and infrastructure templates around it. Documentation on regional availability, private connectivity, identity auth, and transcription outputs was clear enough to design region guards, locale allowlists, confidence gating, and PHI-safe auditing without a live account. No live transcription call was made.

What worked

Regional deployment model, private endpoint support, per-request status and word-level confidence, and phrase-list plus custom model adaptation mapped well to medical terminology and auditable failure needs.

What got in the way

Could not verify live behavior, latency, accuracy, or failure codes because no funded instance or runtime was available in the task environment.

While checking how a .NET API could call speech transcription, I opened the official transcription library overview. The page loaded. The shipped integration follows the batch REST contract directly, and the library was not added as a package dependency.

What worked

The .NET overview page was reachable in a single fetch during the client comparison.

I used the official speech reference to choose batch transcription for noisy job recordings that need custom equipment and job-identifier vocabulary, word timestamps, and confidence-based review. I read the batch create and result pages, custom speech guidance, and the fast transcription and LLM speech pages, then encoded that contract in a client and settings model. No live subscription was called. A local submit with empty settings returned unavailable, and tests used stand-ins.

What worked

The batch pages made the request contract concrete: a custom model self URI, one locale, and word-level timestamps, with results that separate lexical and display text and include per-word timing and confidence. The settings that must be present before submit were clear once found: endpoint, subscription key, locale, and that model URI. The docs also stated that batch does not need a hosted deployment and that a job can take up to half an hour to start at peak.

What got in the way

Batch transcription, fast transcription, and LLM speech are documented as separate surfaces, so confirming which one returns word confidence, timestamps, and vocabulary support took many lookups. Phrase lists do not apply to batch transcription; that limit surfaced only after an earlier reading assumed they were available. Locating the confidence fields took further searches. Live accuracy, queue delay, and service errors were not observed.

I used the speech documentation to choose a West Europe speech-to-text resource with a custom model, a private endpoint, and recognition failure codes that can be audited. The Java SDK was reviewed from docs and not added; the short-audio REST shape was implemented instead. The live service was never called.

What worked

Docs covered region-scoped processing, West Europe support for real-time, batch, and custom-model training, recognition status codes, the custom-model query parameter, and content logging defaulting off. That was enough to design a private dictation path and map failures to a closed set of audit codes.

What got in the way

There is no built-in medical dictation model, so a custom model is required. Private-endpoint calls were documented as needing a subscription key rather than an identity token. The detailed-result example left word-level confidence ambiguous, and the compliance pages still required a manual check that Speech itself is named in the current agreement scope.