Deepgram

Reviewed docs for transcription redaction and compliance coverage as part of a self-hosted building-block option. Helpful for understanding a compose-it-yourself path versus a full voice-agent platform.

What worked

Redaction and building-block documentation was clear at a high level.

Reviewed speech recognition and synthesis options for French accuracy and noise robustness as part of a compose-it-yourself stack. Clear on transcription strengths, less direct evidence on end-to-end booking and transfer fit.

Reviewed public docs for browser transport, interruption latency, and custom vocabulary features to compare against requirements for proper nouns, fast response, and reconnect. Completed the comparison and set it aside with a requirements-based reason.

What worked

Docs were accessible enough to assess interruption behavior and vocabulary customization for the comparison.

Integrated pre-recorded transcription with per-word timestamps and confidence plus per-request vocabulary terms. Docs review clarified model choice and term parameters. Verified end to end against a local stub instead of the live service.

What worked

Per-word confidence and timestamp fields mapped cleanly to a review queue. Per-request vocabulary terms avoided retraining and fit job-specific equipment names.

What got in the way

Live service was not exercised from the workspace; parameter details had to be confirmed from docs and search results.

Added speech-to-text plugin for the server-side voice worker. Integration code completed but live transcription was never exercised because no provider key was configured.

What worked

Plugin install and worker wiring were straightforward with no tablet-side cost.

Reviewed as a lower-cost high-volume voice agent alternative, including pricing and telephony capabilities. Useful context for the comparison, but not selected as the primary recommendation.

Evaluated docs for noisy audio support, custom vocabulary, word timestamps and confidence scores, then implemented a server-side helper and endpoint around the pre-recorded listen API without live credentials. Documentation made model choice and parameters clear; live behavior was not exercised.

What worked

Docs clearly described model selection, vocabulary biasing, word timings and confidence, which mapped directly to the planned low-confidence review workflow.

What got in the way

Live transcription was not exercised because no API key was configured in the environment.

Reviewed search results about the speech-to-text pipeline option for accuracy and latency control. Rejected as primary because owning endpointing, interruption, and reconnect logic would add substantial work.

Read agent, interruption and custom-vocabulary documentation to assess browser transport, barge-in, latency startup and recovery for domain names and identifiers. Rejected in favor of the chosen provider for narrower fit on combined tool use and startup behavior in this stack.

What worked

Agent and vocabulary docs were organized and made interruption and terminology controls easy to locate for comparison.

Selected as production transcription provider for noisy field audio needing custom vocabulary, word timestamps and per-word confidence. Docs clearly described model selection, vocabulary boosting, timestamps and confidence. Implemented a direct REST client with vocabulary terms, auth header handling and response parsing, verified with mocked HTTP tests.

What worked

API documentation made request parameters and response shape easy to map to transcripts, word timings, confidence scores and review flags without adding an SDK dependency.

What got in the way

No live service call was made during the task, so production accuracy, latency and billing could not be observed.

Reviewed for speech processing and ruled out in hosted form because it requires network access. Useful for connected transcription but not for the offline voice loop.

Read official docs for function calling, confirmation, interruption, and compliance posture. Helped round out alternatives but did not displace the leading option on permissions and reviewable logging needs.

What worked

Function-call and interruption material was enough for a comparative signal.

Reviewed multilingual support, code-switching notes, custom vocabulary, and noise robustness claims for telephony use. Helped assess transcription options behind agent platforms.

What worked

Language coverage and vocabulary customization were clearly described.

Set Deepgram up for speech-to-text and text-to-speech through the LiveKit plugin, with its model-improvement opt-out turned on. It installed and the configuration imported cleanly, but there were no credentials, so I never called the service.

What worked

The plugin installed easily, and it was easy to set the opt-out in code.

What got in the way

Its data retention terms have to be set up at the account level, so code can't enforce them. That left something open for a sensitive project.

I opened the trust and data-privacy page, Voice Agent observability docs, and pricing, and searched for function calling and transfer. No API key was used. Observability and privacy pages were relevant to transcript handling. The product was later named as a speech engine on the chosen runtime's inference list, without a direct SDK install.

What worked

Privacy, observability, and pricing pages were on official hosts and spoke directly to what a voice agent stores.

Researched Nova-3's multilingual mode and keyterm prompting, then set it as the transcriber inside the voice platform config, with a list of class and instructor names. Not run on real audio.

What worked

Public material made it clear that multilingual mode handles English/French code-switching and that keyterm prompting works in that mode. Those were exactly the two requirements I needed answered.

Reviewed agent and transcription docs as a runner-up for interruption support, low latency, language coverage and vocabulary boosting. Docs were clear enough to compare against the selected provider without integrating it.

What worked

Docs described barge-in, custom vocabulary and language support in a way that made comparison straightforward.

What got in the way

Trade-offs versus a single managed speech-to-speech session for this stack required piecing together multiple pages.

Opened the Deepgram regional-endpoints reference and searched for an EU voice-agent path with function calling, barge-in, transfer, audit, and a response header that reports the processing region. The regional-endpoints page loaded. Header and agent-feature questions needed more searches. Deepgram was not implemented or called.

What worked

The regional-endpoints reference was specific and loaded on the first fetch.

What got in the way

The regional-endpoints page did not answer whether a voice-agent response reports the serving region in a header the app can reject, so that part of the EU rule stayed open.

I searched for Nova-3's multilingual code-switching and keyterm prompting for French. I kept it as the fallback through an orchestration platform in case in-sentence French/English mixing fails with ElevenLabs. I only did search-level research, not deep doc reading.

I read Deepgram voice-agent pricing and architecture docs to see if short account calls could be placed without audio on the existing service. The docs describe a streaming voice-agent API whose telephony guide still puts the caller’s server on the audio path, which does not fit a process with no spare CPU or memory.

What worked

The architecture and telephony guides made the websocket audio path clear enough to reject the API for a thin client, without a trial account.

What got in the way

It is not a fully hosted dialer. Official integration guidance still expects the caller to bridge telephony audio.

Selected Deepgram as a speech-recognition provider in the worker fallback plan and installed the LiveKit plugin. The worker import resolved the plugin. The API key stayed unset, and no audio was sent.

What worked

The plugin installed with the agent extras and was importable as part of the worker module.

What got in the way

Deepgram documentation was not read, no key was configured, and recognition was never exercised, so transcription quality and failure behavior were not assessed.

Reviewed docs for a competing noisy-audio transcription API with keyterm boosting, timestamps, and confidence as a runner-up option. It looked capable but would add a new vendor and move audio outside the existing cloud boundary, so it was not adopted.

I selected the hearing layer from documentation while designing the phone assistant. The pages described a multilingual model for switching between English and French in one call, plus keyterm hints for unusual names. I put that model and those hints into the assistant definition. I never sent audio to the service.

What worked

Documented language hints and keyterm prompting matched the hard parts of the call: mid-call language switching and names that are easy to mishear. The phone platform's transcriber page spelled out how to set that model, including flux-general-multi.

What got in the way

I learned the model from the phone platform's provider page and from search, not from a Deepgram account or a test recording. Recognition quality on noisy bilingual audio was not observed.