Overview
GoogleDecisionsClassifier answers classifier questions through Google’s Decisions API. All the questions about one state go to Google in one request. Structured state, instructions, and descriptions are sent as JSON. Besides text, it can see images, hear clips of audio, and read files such as PDFs and videos.
A GoogleDecisionsClassifier asks through a GoogleDecisionsClient, which holds the HTTP/2 connection, retries when Google is busy, and counts tokens. Build the classifier with an API key to give it a client of its own, or pass a GoogleDecisionsClient to share one connection between several classifiers.
Classifier Questions
Ask yes/no, choice, and score questions about a call transcript. Run with
--google.Questions About an Image
Ask about an image, such as a frame from a camera. Run with
--google.Installation
Prerequisites
A Gemini API key, usually set as an environment variable:Configuration
GoogleDecisionsClassifier
Gemini API key, when the classifier should have a client of its own. One of
api_key and client is required.A client to share with other classifiers. One of
api_key and client is
required.Seconds a client of its own waits for an answer, retries included, before
raising
ClassifierError.GoogleDecisionsClient
How many times to retry a request that failed for a reason that may pass, such
as Google being busy, with backoff.
Usage
Basic Usage
VoicemailDetector or UIWorker.
Sharing a Client
Several classifiers can share oneGoogleDecisionsClient, and with it one connection pool and one token count:
cleanup(). A shared client is left open, so close it yourself with await client.close() when every classifier using it is done.
Asking About Images, Audio, and Files
Pass images, clips of audio, and files ininputs= to ask(), yes_no(), choice() or score(), as ClassifierImage, ClassifierAudio and ClassifierFile objects:
audio_sample_rate is 16000. Files are sent as bytes or, for a ClassifierFile with a url, by URL.
Opening the Connection Early
setup() opens the connection to Google ahead of the first question, so the first answer does not pay for the TLS handshake. Components that own a classifier, such as VoicemailDetector, call it for you. If the connection cannot be opened at setup, a warning is logged and the first question opens it instead.
GoogleDecisionsClassifier Properties
GoogleDecisionsClient Reference
Properties
Methods
connect
ClassifierError if Google could not be reached or refused the request.
close
Metrics
After every call, aGoogleDecisionsClassifier reports the time it took as ProcessingMetricsData and the tokens used as LLMUsageMetricsData through its on_metrics event.
Notes
- Limits: a choice question takes at most 255 options, available as
GOOGLE_DECISIONS_MAX_CHOICE_OPTIONSinpipecat.classifiers.google.decisions.classifier. A question with more raisesClassifierErrorbefore anything is sent. - File types: the MIME types of files it reads are listed in
GOOGLE_DECISIONS_FILE_TYPESin the same module. A file of another type raisesClassifierError. - Errors: a rejected request, a busy Google after every retry, an unreachable server, an input Google’s filters blocked, or a reply missing an answer all raise
ClassifierError. - Idle connections are kept open for 240 seconds, so a gap between questions does not cost a new TLS handshake.