Chose the Realtime API as the primary speech provider and implemented a server endpoint that mints short-lived client secrets plus a browser WebRTC client with tools, transcription hints and barge-in. No API key was available, so the integration was written from the docs and never run against the live service.
- What worked
- The WebRTC guide and the conversations guide described the client-secret flow, the session config (instructions, input transcription with language and prompt, turn detection, tools) and the event names clearly enough to write the whole integration. Ephemeral keys keep the real key on the server.
- What got in the way
- There is no built-in session resumption, so I had to build my own reconnect logic that replays a transcript into a fresh session. Model names were inconsistent across docs pages and third-party sites, so I needed several extra fetches to confirm which models actually exist. It was unclear whether tool schemas reliably accept nullable union types, so I avoided them.
