Overview
ZoomAISTTService provides real-time speech recognition with the Scribe API from Zoom AI Services, streaming audio over a WebSocket in Live mode. By default, Scribe detects when each user turn ends, so the service drives turn detection for live streaming without a local VAD. With vad_force_turn_endpoint=True, Pipecat’s VAD and turn strategies decide instead. It supports Scribe’s additional features, including English Pro model, multilingual models, language detection, speaker diarization, custom vocabulary, and text polishing, etc.
Zoom AI STT API Reference
Pipecat’s API methods for Zoom AI STT integration
Example Implementation
Complete voice agent with Scribe API
Scribe Documentation
Official Zoom Scribe API documentation
Zoom AI Services
Get credentials and manage your Zoom AI Services plan
Installation
To use Zoom AI services, install the required dependency:Prerequisites
Zoom AI Account Setup
Before using Zoom AI STT, you need:- Zoom AI Services access: Sign up through Zoom AI Services
- API key: Create a Zoom AI Services API key. See Get credentials.
Required Environment Variables
ZOOM_AI_API_KEY: Your Zoom AI Services API key
Configuration
ZoomAISTTService
Zoom AI Services API key. It is sent on every connection and reconnect, so it
must stay valid for as long as the service runs.
Scribe Live WebSocket URL. A plaintext
ws:// URL to a host other than
localhost logs a warning, since the API key and audio are sent unencrypted.Input audio sample rate in Hz. If
None, uses the pipeline’s audio sample
rate. Audio is resampled to 16 kHz for Scribe.Whether a user turn started by Scribe interrupts the bot. Only applies with
Scribe turn detection (
vad_force_turn_endpoint=False).Selects the turn detection mode. When
False, Scribe’s voice activity
detection ends each turn. When True, Scribe’s voice activity detection is
off and each VADUserStoppedSpeakingFrame commits the turn, so Pipecat’s VAD
and user turn strategies (such as Smart Turn) decide when the user is done.
Requires a local VAD analyzer. See With Pipecat Turn
Detection.Runtime-configurable settings for the STT service. See Settings
below.
P99 latency from speech end to final transcript, in seconds. Used with
vad_force_turn_endpoint=True; override it for your deployment. See the STT
benchmark.Settings
Runtime-configurable settings passed via thesettings constructor argument using ZoomAISTTService.Settings(...). Settings map to the Scribe API fields of the same name; see the Scribe documentation for what each feature does and its availability. See Service Settings for how settings work in Pipecat.
Usage
Basic Setup
Configuring Scribe Features
Scribe features are set throughSettings. This example uses the English Pro model, adds a custom vocabulary, and turns on speaker diarization:
TranscriptionFrame carries Scribe’s label (for example "Speaker 1") in speaker.label. A transcript in which the speaker changed is pushed as one frame per speaker. When text polishing rewords a transcript too much to split it, the frame has no speaker. Scribe’s event is in result.
See Select a model, Custom vocabulary, and the Scribe documentation for what each feature does.
With the Multilingual Model
For callers who may switch languages, usezoom-scribe-multilingual with continuous language detection. language is the language Scribe expects first. Each TranscriptionFrame has the language Scribe detected for that transcript in language.
With Pipecat Turn Detection
Setvad_force_turn_endpoint=True to turn off Scribe’s voice activity detection and let Pipecat’s VAD and turn strategies decide when the user is done. Each VADUserStoppedSpeakingFrame commits the turn, and Scribe returns its final transcript. The service keeps the default user turn strategies, so the user aggregator needs a VAD analyzer:
Updating Settings Mid-Stream
Send anSTTUpdateSettingsFrame to change settings during a conversation. The service reconnects with a new Scribe session once the current turn ends, except for speaker_format, which applies without reconnecting:
Notes
- Final transcripts only: The service pushes a final transcript per turn, with no interim results.
- Idle sessions: Scribe ends a session after 30 seconds without audio, so the service sends keepalive silence when input stops (see Session limits). Keepalive silence and the silence sent to end a turn count toward usage.
- Errors: If Zoom refuses the connection (for example, an invalid API key), the error includes Zoom’s reason. If Scribe rejects the session’s settings, the error has the
INVALID_REQUESTcategory, since a new session would be rejected the same way. Errors include the session’s tracking ID (x-zm-trackingid), which Zoom support uses to find a session. See Error Handling. - Time to final segment (TTFS): With Scribe turn detection, Scribe defines the turn boundary, so the service reports no TTFS metric. With
vad_force_turn_endpoint=True, it reports TTFS usingttfs_p99_latency.