> ## Documentation Index
> Fetch the complete documentation index at: https://docs.pipecat.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Zoom AI Voice Agent

> Speech-to-speech conversation with ZoomAIRealtimeLLMService, built on the Zoom AI Voice Agent API in Live mode

## Overview

`ZoomAIRealtimeLLMService` provides speech-to-speech conversation with the [Voice Agent](https://developers.zoom.us/docs/ai-services/voice-agent/live-mode/) from Zoom AI Services. The Voice Agent runs the whole voice loop on Zoom's side: turn detection, speech recognition, the model, and speech synthesis. The service replaces STT, the LLM, and TTS in a pipeline, and supports function calling, interruptions, and transcripts like [OpenAI Realtime](https://docs.pipecat.ai/api-reference/server/services/s2s/openai.md). The Voice Agent detects turns and handles barge-in itself, so the pipeline needs no VAD analyzer.

> [card] [Zoom AI Voice Agent API Reference](https://reference-server.pipecat.ai/en/latest/api/pipecat.services.zoom_ai.realtime.llm.html)
>
> Pipecat's API methods for Zoom AI Voice Agent integration

> [card] [Example Implementation](https://github.com/pipecat-ai/pipecat/blob/main/examples/realtime/realtime-zoom-ai.py)
>
> Complete Zoom AI Voice Agent conversation example

> [card] [Voice Agent Documentation](https://developers.zoom.us/docs/ai-services/voice-agent/live-mode/)
>
> Official Zoom Voice Agent API documentation

> [card] [Zoom AI Services](https://www.zoom.ai/)
>
> Get credentials and manage your Zoom AI Services plan

## Installation

```bash theme={null}
uv add "pipecat-ai[zoom-ai]"
```

## Prerequisites

### Zoom AI Account Setup

1. **Zoom AI Services access**: Sign up through [Zoom AI Services](https://www.zoom.ai/).
2. **API key**: Create a Zoom AI Services API key. See [Get credentials](https://developers.zoom.us/docs/ai-services/auth/).

### Required Environment Variables

* `ZOOM_AI_API_KEY`: Your Zoom AI Services API key

## Configuration

### ZoomAIRealtimeLLMService

**api\_key** `str` *required*

Zoom AI Services API key. It is sent on every connection and reconnect, so it
must stay valid for as long as the service runs.

**base\_url** `str` (default: wss\://api.zoom.us/v2/ws/aiservices/voice-agent/live)

Voice Agent WebSocket URL.

**settings** `ZoomAIRealtimeLLMService.Settings` (default: None)

Runtime-configurable settings. See [Settings](#settings) below.

**start\_audio\_paused** `bool` (default: False)

Whether to start with audio input paused.

### Settings

Runtime-configurable settings passed via the `settings` constructor argument using `ZoomAIRealtimeLLMService.Settings(...)`. They map to the Voice Agent's [session settings](https://developers.zoom.us/docs/ai-services/voice-agent/live-mode/#configure-the-session) and can be updated mid-conversation with `LLMUpdateSettingsFrame`. See [Service Settings](https://docs.pipecat.ai/pipecat/fundamentals/service-settings.md) for details.

| Parameter | Type | Default | Description |
| - | - | - | - |
| `model` | `str` | `"gemma-4"` | Model identifier: a Zoom model, or your `provider`'s model when the provider sets none of its own. Fixed once the session is configured. See [Think settings](https://developers.zoom.us/docs/ai-services/voice-agent/live-mode/#think-settings). *(Inherited from base settings.)* |
| `system_instruction` | `str \| None` | `None` | System instruction/prompt. *(Inherited from base settings.)* |
| `session_properties` | `SessionProperties` | `SessionProperties()` | OpenAI Realtime session properties. The service uses `instructions`, `tools`, `tool_choice`, `max_output_tokens`, `reasoning`, and `audio.output.voice` (when `speak` sets no voice). |
| `provider` | `Provider \| None` | `None` | Where the model runs: Zoom's own models (`None`), or your own OpenAI or Anthropic account. Fixed once the session is configured. See [Provider](#provider). |
| `listen` | `Listen \| None` | `None` | How the agent listens: the caller's language and custom vocabulary. `None` takes the server's defaults. Fixed once the session is configured. See [Listen](#listen). |
| `speak` | `Speak \| None` | `None` | How the agent speaks: its voice. `None` takes the server's default voice. See [Speak](#speak). |
| `extra` | `dict[str, Any]` | `{}` | Voice Agent session fields the service has no setting for, shaped like the `session` of `session.update`. Merged into each session update over the settings above. |

`provider`, `listen`, and `speak` also accept plain dicts in the same shape, such as `{"voice": "ava"}`.

### Provider

```python theme={null}
from pipecat.services.zoom_ai.realtime.events import Provider
```

| Parameter | Type | Default | Description |
| - | - | - | - |
| `type` | `Literal["zoom", "openai", "anthropic"]` | `"zoom"` | `"zoom"` for a Zoom-hosted model, or `"openai"` / `"anthropic"` to use your own key for that provider's model. |
| `model` | `str \| None` | `None` | Model ID from the provider. `None` uses the service's `model` setting. |
| `headers` | `dict[str, str] \| None` | `None` | Your provider API key, as `{"X-Api-Key": "..."}`. Required for `"openai"` and `"anthropic"`. This is your OpenAI or Anthropic key, not your Zoom AI Services key. |
| `metadata` | `dict[str, Any] \| None` | `None` | The provider model's limits, such as `context_window` and `max_output_tokens`. |

See [Bring your own key](https://developers.zoom.us/docs/ai-services/voice-agent/live-mode/#bring-your-own-key) for the providers and models the Voice Agent supports.

### Listen

```python theme={null}
from pipecat.services.zoom_ai.realtime.events import Listen, Transcription, Vocabulary
```

| Parameter | Type | Default | Description |
| - | - | - | - |
| `transcription.language` | `str \| None` | `None` | Language of the caller, for example `"en"`. See [Listen settings](https://developers.zoom.us/docs/ai-services/voice-agent/live-mode/#listen-settings) for supported languages. |
| `transcription.vocabulary` | `Vocabulary \| None` | `None` | Custom vocabulary: `Vocabulary(phrases=[...])`, names and terms to recognize, such as product names. |

### Speak

```python theme={null}
from pipecat.services.zoom_ai.realtime.events import Speak
```

| Parameter | Type | Default | Description |
| - | - | - | - |
| `voice` | `str \| None` | `None` | Voice ID, for example `"jenna"` (the server's default). See [Available voices](https://developers.zoom.us/docs/ai-services/voice-agent/live-mode/#available-voices). |

## Usage

### Basic Setup

```python theme={null}
import os
from pipecat.services.zoom_ai.realtime.llm import ZoomAIRealtimeLLMService

llm = ZoomAIRealtimeLLMService(
    api_key=os.getenv("ZOOM_AI_API_KEY"),
    settings=ZoomAIRealtimeLLMService.Settings(
        system_instruction="You are a helpful assistant.",
    ),
)
```

Register tools on the `LLMContext`, as for other LLM services. See [Function Calling](https://docs.pipecat.ai/pipecat/learn/function-calling.md).

### Choosing a Voice and Vocabulary

```python theme={null}
from pipecat.services.zoom_ai.realtime.events import (
    Listen,
    Speak,
    Transcription,
    Vocabulary,
)

llm = ZoomAIRealtimeLLMService(
    api_key=os.getenv("ZOOM_AI_API_KEY"),
    settings=ZoomAIRealtimeLLMService.Settings(
        system_instruction="You are a helpful assistant.",
        speak=Speak(voice="jenna"),
        listen=Listen(
            transcription=Transcription(
                language="en",
                vocabulary=Vocabulary(phrases=["Pipecat"]),
            )
        ),
    ),
)
```

### Using Your Own OpenAI or Anthropic Model

```python theme={null}
from pipecat.services.zoom_ai.realtime.events import Provider

llm = ZoomAIRealtimeLLMService(
    api_key=os.getenv("ZOOM_AI_API_KEY"),
    settings=ZoomAIRealtimeLLMService.Settings(
        model="gpt-4o",
        provider=Provider(
            type="openai",
            headers={"X-Api-Key": os.getenv("OPENAI_API_KEY")},
        ),
    ),
)
```

The Voice Agent still transcribes the caller and speaks the reply; only the model's turn uses your provider account.

### Updating Settings Mid-Conversation

Send an `LLMUpdateSettingsFrame` to change the instructions, tools, or voice during a conversation:

```python theme={null}
from pipecat.frames.frames import LLMUpdateSettingsFrame
from pipecat.services.zoom_ai.realtime.events import Speak

await worker.queue_frame(
    LLMUpdateSettingsFrame(
        delta=ZoomAIRealtimeLLMService.Settings(
            system_instruction="You are now a shipping assistant.",
            speak=Speak(voice="ava"),
        )
    )
)
```

## Notes

* **Turn detection**: The Voice Agent detects turns, noise, and barge-in itself. None of these is configurable.
* **Settings fixed at connection**: `model`, `provider`, and `listen` are fixed once the session is configured. Updating them mid-conversation logs a warning and applies to the next session. Updates to instructions, tools, and voice take effect immediately.
* **Audio**: Input audio is sent at the pipeline's input sample rate. The agent speaks 16 kHz audio, played at 24 kHz. Audio that arrives while the session is connecting or reconnecting is held (up to the latest three seconds) and sent once the session is ready.
* **Transcripts**: The caller's speech is pushed upstream as `TranscriptionFrame`s, and the agent's speech downstream as `TTSTextFrame`s.
* **Text-only replies**: When the audio output modality is disabled, the agent replies in text only.
* **Reconnection**: A session lasts up to 60 minutes. If the session ends unexpectedly (the connection closing, or a session reaching its time limit), the service reconnects, up to 3 times in a row, and restores the conversation. If reconnecting fails, it pushes a permanent error.
* **Errors**: If Zoom refuses the connection (an invalid API key, for example), the error includes Zoom's reason. Errors the Voice Agent reports without ending the session are pushed as non-fatal errors. If it rejects the session's settings (an unknown voice, say) or the provider rejects your provider key, the error is permanent. Errors from the Voice Agent include the session's tracking ID (`x-zm-trackingid`), which Zoom support uses to find a session. See [Error Handling](https://docs.pipecat.ai/pipecat/fundamentals/error-handling.md).
* **Function call output**: A function call's output is created right after the call in the conversation, where the Voice Agent's model reads it.
* **System instruction precedence**: The `system_instruction` from service settings takes precedence over an initial system message in the LLM context. A warning is logged when both are set.

## Event Handlers

| Event | Description |
| - | - |
| `on_conversation_item_created` | Called when a new conversation item is created in the session |
| `on_conversation_item_updated` | Called when a conversation item is updated or completed. `item` is `None` when a user audio transcription completes. |

```python theme={null}
@llm.event_handler("on_conversation_item_created")
async def on_item_created(service, item_id, item):
    print(f"New conversation item: {item_id}")

@llm.event_handler("on_conversation_item_updated")
async def on_item_updated(service, item_id, item):
    print(f"Conversation item updated: {item_id}")
```


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.