Skip to main content

Overview

GoogleDecisionsClassifier answers classifier questions through Google’s Decisions API. All the questions about one state go to Google in one request. Structured state, instructions, and descriptions are sent as JSON. Besides text, it can see images, hear clips of audio, and read files such as PDFs and videos. A GoogleDecisionsClassifier asks through a GoogleDecisionsClient, which holds the HTTP/2 connection, retries when Google is busy, and counts tokens. Build the classifier with an API key to give it a client of its own, or pass a GoogleDecisionsClient to share one connection between several classifiers.

Classifier Questions

Ask yes/no, choice, and score questions about a call transcript. Run with --google.

Questions About an Image

Ask about an image, such as a frame from a camera. Run with --google.

Installation

Prerequisites

A Gemini API key, usually set as an environment variable:

Configuration

GoogleDecisionsClassifier

str | None
default:"None"
Gemini API key, when the classifier should have a client of its own. One of api_key and client is required.
GoogleDecisionsClient | None
default:"None"
A client to share with other classifiers. One of api_key and client is required.
str
default:"gemini-decision-1"
The decision model a client of its own asks.
float
default:"10.0"
Seconds a client of its own waits for an answer, retries included, before raising ClassifierError.
str | None
default:"None"
Name of the classifier, as it appears in logs and metrics.

GoogleDecisionsClient

str
required
Gemini API key.
str
default:"gemini-decision-1"
The decision model to ask.
float
default:"10.0"
Seconds to wait for an answer, retries included, before raising ClassifierError.
int
default:"3"
How many times to retry a request that failed for a reason that may pass, such as Google being busy, with backoff.

Usage

Basic Usage

Most of the time you don’t call the classifier yourself: you pass it to a component that asks it, such as VoicemailDetector or UIWorker.

Sharing a Client

Several classifiers can share one GoogleDecisionsClient, and with it one connection pool and one token count:
A client the classifier created is closed in the classifier’s cleanup(). A shared client is left open, so close it yourself with await client.close() when every classifier using it is done.

Asking About Images, Audio, and Files

Pass images, clips of audio, and files in inputs= to ask(), yes_no(), choice() or score(), as ClassifierImage, ClassifierAudio and ClassifierFile objects:
The inputs are sent as base64 data after the state, in the order given. Clips of audio are sent as 16 kHz WAV, so audio_sample_rate is 16000. Files are sent as bytes or, for a ClassifierFile with a url, by URL.

Opening the Connection Early

setup() opens the connection to Google ahead of the first question, so the first answer does not pay for the TLS handshake. Components that own a classifier, such as VoicemailDetector, call it for you. If the connection cannot be opened at setup, a warning is logged and the first question opens it instead.

GoogleDecisionsClassifier Properties

GoogleDecisionsClient Reference

Properties

Methods

connect

Opens the connection to Google by looking up the model, a cheap request that also checks the key and its access to the model. It connects once: a client shared by several classifiers is connected by each of them, and only the first call sends anything. Raises ClassifierError if Google could not be reached or refused the request.

close

Closes the connection pool.

Metrics

After every call, a GoogleDecisionsClassifier reports the time it took as ProcessingMetricsData and the tokens used as LLMUsageMetricsData through its on_metrics event.

Notes

  • Limits: a choice question takes at most 255 options, available as GOOGLE_DECISIONS_MAX_CHOICE_OPTIONS in pipecat.classifiers.google.decisions.classifier. A question with more raises ClassifierError before anything is sent.
  • File types: the MIME types of files it reads are listed in GOOGLE_DECISIONS_FILE_TYPES in the same module. A file of another type raises ClassifierError.
  • Errors: a rejected request, a busy Google after every retry, an unreachable server, an input Google’s filters blocked, or a reply missing an answer all raise ClassifierError.
  • Idle connections are kept open for 240 seconds, so a gap between questions does not cost a new TLS handshake.