Skip to main content
Voice Agents

Managed Voice Agent

Stream audio over a WebSocket and get a speaking agent back. Two protocols: CallMissed-native and Deepgram Voice Agent compatible.

Overview

The Managed Voice Agent is a full speech-to-speech pipeline behind a single WebSocket. You stream microphone audio in; you get synthesized speech and conversation events back. Speech recognition, the language model, text-to-speech, turn-taking and interruption handling are all run and tuned for you.

Unlike the Voice Session API, there is no WebRTC and no client SDK to install — a plain WebSocket and raw PCM is the whole integration.

Two wire protocols are served, backed by the same engine:

EndpointProtocol
wss://api.callmissed.com/v2/voice/agentCallMissed-native
wss://api.callmissed.com/v1/agent/converseDeepgram Voice Agent compatible

If you already have an integration written against Deepgram's Voice Agent API, point it at the second URL and it will work unchanged.

Both live on api.callmissed.com, the same host as the rest of the API — there is no separate hostname to allowlist.

Authentication

Both endpoints take an API key with stt, tts and llm permissions.

Authorization: Token cm_your_api_key

Authorization: Bearer cm_your_api_key is accepted too.

Browsers cannot set headers on a WebSocket, so the subprotocol form is also accepted:

new WebSocket(url, ["token", "cm_your_api_key"])

A missing or invalid key, a key without those permissions, an exhausted plan limit, the browser/API voice sessions at once limit and a credit balance below the minimum are all checked before the socket is accepted, so a refused connection fails at the handshake rather than mid-conversation. A refused handshake fails the WebSocket upgrade with HTTP 403 whatever the reason; no Error frame is sent, because the socket never opened. Check the key, its permissions and your balance first.

Session flow

  1. Connect. The server sends Welcome with a request_id.
  2. Send Settings as your first message, within 15 seconds. Anything else first (including audio) ends the session with an Error.
  3. Wait for SettingsApplied, then start streaming audio.
  4. Send raw audio as binary frames; receive synthesized audio as binary frames and events as JSON on the same socket.
{ "type": "Welcome", "request_id": "fc553ec9-5874-49ca-a47c-b670d525a4b1" }

Settings (native)

The native shape is organised around what you actually choose: one model id per role.

{
  "type": "Settings",
  "audio": {
    "input":  { "encoding": "linear16", "sample_rate": 24000 },
    "output": { "encoding": "linear16", "sample_rate": 24000 }
  },
  "agent": {
    "prompt": "You are a concise support agent for an Indian retail brand.",
    "greeting": "Hi, how can I help?",
    "language": "en-IN",
    "llm": { "model": "gpt-oss-120b", "temperature": 0.4 },
    "stt": { "model": "saaras:v4" },
    "tts": { "model": "bulbul:v3", "voice": "shubh" },
    "tools": [
      {
        "name": "lookup_order",
        "description": "Look up an order by id",
        "parameters": {
          "type": "object",
          "properties": { "order_id": { "type": "string" } },
          "required": ["order_id"]
        }
      }
    ]
  },
  "tags": ["support"]
}
FieldDefaultNotes
audio.input / audio.outputlinear16 at 24000 HzSee Audio format
agent.prompta concise built-in assistant promptString. Longer than 25,000 characters is truncated, with a PROMPT_TOO_LONG warning
agent.greeting—The first line the agent speaks
agent.languageen-INBCP-47. Falls back to agent.stt.language, then agent.tts.language
agent.llm.modelgemma-4-31bAny LLM id from GET /api/v1/voice/models except deepgram-voice-* (see Choosing models)
agent.llm.temperaturemodel defaultNumber, 0–2
agent.stt.modelsaaras:v4
agent.tts.modelplan-dependentFree plan bulbul:v3; paid plans (starter and above) sonic-3.6
agent.tts.voicemodel defaultWith no agent.tts.model: free plan shubh, paid plans skylar. With a model set, that model's own default speaker. A voice you set must be a speaker of the chosen model
agent.tools[]Functions the agent may call: name (required), description, parameters (JSON Schema)
agent.record_callsfalseBoolean. Records the session as one MP3 with both sides; fetch it afterwards with GET /v1/voice/sessions/{id}/recording. The session is listed by GET /v1/voice/sessions. Transcripts are saved either way
agent.recording_notice_enabledtrueBoolean. With recording on, the agent first says a short notice that the call is recorded; false skips it
agent.recording_noticea short default noticeString, at most 300 characters. The notice's wording
agent.context.messages[]Earlier turns to resume from, in the History shape
agent.nudge_max0 (off)Integer, 0–5. How many times the agent prompts a silent caller. See Caller silence
agent.nudge_delay_seconds12Number, 3–120. Seconds of silence before each prompt
agent.nudge_message"Are you still there?" in the session languageString. The prompt's wording
agent.nudge_hanguptrueBoolean. End the session after the last unanswered prompt
flags.historyfalseBoolean. Send a History event for every turn and completed function call
tags[]Array of strings

agent.llm, agent.stt and agent.tts also accept a bare model id string as shorthand, e.g. "llm": "kimi-k2.5". Model ids, voice, language and greeting must be strings; any other type fails Settings with INVALID_SETTINGS.

A model id the voice agent cannot serve fails Settings with an INVALID_SETTINGS error naming it; it is never swapped for another model. If a servable stack still cannot start (for example, a voice that is not a speaker of the chosen model), the session runs on a backup stack. You get a STACK_SUBSTITUTED warning naming the models in use, and the session is billed for those models.

Settings (Deepgram-compatible)

On /v1/agent/converse, send Deepgram's Voice Agent Settings shape unchanged, with CallMissed model ids in agent.listen.provider.model, agent.think.provider.model and agent.speak.provider.model (model_id is accepted too).

Deepgram's own model names are accepted where CallMissed serves the same model, in Settings and in UpdateListen / UpdateSpeak:

You send (Deepgram)Runs as
listen: a Deepgram speech-to-text model, e.g. flux-general-en, flux-general-multi, nova-3-medical, nova-2-phonecalldeepgram-<model>, the same Deepgram model
speak: an Aura-2 model, e.g. aura-2-thalia-endeepgram-aura-2 with voice thalia
speak: an Aura-1 model, e.g. aura-asteria-endeepgram-aura-1 with voice asteria
think: an OpenAI model CallMissed serves under the same name, e.g. gpt-6-sol, gpt-6-luna, gpt-5.6-terraunchanged

An id that is already a CallMissed id is never re-routed (deepgram-nova-3 stays deepgram-nova-3). Anything else, including think models CallMissed does not serve (for example gpt-4o-mini or Anthropic and Google models) and Flux TTS voices (flux-<voice>-<lang>), fails Settings with INVALID_SETTINGS naming the id. It is never swapped for a similar model. agent.think.prompt, agent.think.functions, agent.greeting, the audio block and tags behave as on the native surface, and so do agent.record_calls, agent.recording_notice_enabled and agent.recording_notice (CallMissed additions; Deepgram has no equivalent).

Also supported as on Deepgram: mulaw and alaw audio (see Audio format), resuming with agent.context.messages and the History events (on by default; flags.history: false turns them off, see Resuming a conversation), ForceEndTurn, and fallback arrays in agent.think and agent.speak (see Fallback providers).

Differences from Deepgram, each reported rather than silently dropped:

  • Fields accepted but not applied (for example listen.provider.keyterms, speak.provider.speed, think.provider.credentials, audio.output.bitrate) are listed in one SETTINGS_FIELDS_IGNORED warning.
  • audio.output.container must be none (raw frames).
  • A function with a server-side endpoint is dispatched to your client instead (see Tool calling).
  • ForceEndTurn works with every speech-to-text model, not only Flux, so FORCE_END_TURN_UNSUPPORTED is never sent.
  • LatencyReport reports the language-model stage as ttt_token_latency; ttt_text_latency, ttt_tool_latency and ttt_thinking_latency are not measured and are omitted. AgentStartedSpeaking always carries total_latency, tts_latency and ttt_latency, with 0 for a stage that was not measured (for example the greeting), because Deepgram's schema requires all three.

Not supported on /v1/agent/converse:

  • Compressed audio encodings (opus, ogg-opus, mp3, flac, aac, amr-nb, amr-wb, speex, g729) and wav / ogg output containers.
  • A reusable agent configuration id in place of the agent object.
  • defer_until_eot on functions.
  • Multilingual ConversationText language fields.

The update, injection, tool-response and keepalive messages are the same on both protocols, with two differences on /v1/agent/converse, both matching Deepgram: UpdatePrompt adds to the current prompt instead of replacing it, and a second Settings is a fatal SETTINGS_ALREADY_APPLIED error. On the native endpoint a second Settings gets a DUPLICATE_SETTINGS warning and the session continues.

Audio format

Mono, raw frames, in either direction:

encodingDirectionsample_rate (Hz)Default rate
linear16 (16-bit PCM, little-endian)input and output8000, 16000, 24000, 32000, 4800024000
mulaw (G.711 μ-law)input and output8000, 160008000
alaw (G.711 A-law)input and output8000, 160008000
linear32 (32-bit float PCM, little-endian)input only8000, 16000, 24000, 32000, 4800024000

For a phone line (for example a Twilio Media Stream), use mulaw at 8000 Hz in both directions and relay the frames unchanged.

{
  "type": "Settings",
  "audio": {
    "input":  { "encoding": "mulaw", "sample_rate": 8000 },
    "output": { "encoding": "mulaw", "sample_rate": 8000, "container": "none" }
  }
}

Output is raw frames with no container: a WAV or OGG header would be read as audio by most telephony consumers.

An unsupported encoding, sample rate or container is rejected at Settings time on both endpoints, rather than accepted and quietly changed, so you find out at the handshake instead of hearing the wrong thing on a call.

Send audio in small frames, as it is captured (20–100 ms per frame is typical). A single frame carrying more than 5 seconds of audio at your input sample rate is dropped with an AUDIO_CHUNK_TOO_LARGE warning. Send whole samples per frame (an even number of bytes for linear16, a multiple of 4 for linear32): a trailing partial sample is discarded.

Resuming a conversation

Pass earlier turns in agent.context.messages and the agent continues as if they had just happened. Each entry is either a turn or a completed function call, in the same shape as the History events a session sends:

{
  "type": "Settings",
  "agent": {
    "context": {
      "messages": [
        { "type": "History", "role": "user", "content": "What's the weather in New York?" },
        {
          "type": "History",
          "function_calls": [{
            "id": "fc_weather_12345",
            "name": "get_weather",
            "client_side": true,
            "arguments": "{\"location\": \"New York\"}",
            "response": "Partly cloudy, 22°C."
          }]
        },
        { "type": "History", "role": "assistant", "content": "It's partly cloudy, around 22 degrees." }
      ]
    }
  }
}

role is user or assistant. A function call needs id and name (letters, digits, _ and -, at most 64 characters), arguments (a JSON string) and response; thought_signature is kept unchanged when present. A malformed entry fails Settings with INVALID_SETTINGS. At most 100 entries and 60,000 characters are kept; past that the oldest entries are dropped with a CONTEXT_TRUNCATED warning.

With history reporting on (flags.history, default true on /v1/agent/converse and false on the native endpoint), every ConversationText is followed by a History event with the same role and content, and every completed function call produces a History event with its function_calls entry. Save them, and you can resume a session past the 2-hour cap by sending them back as agent.context.messages.

{ "type": "History", "role": "assistant", "content": "It's partly cloudy, around 22 degrees." }

Fallback providers

On /v1/agent/converse, agent.think and agent.speak can each be an array. The first entry configures the session (it carries prompt and functions); up to three more are fallbacks, tried in order when the one serving fails:

{
  "agent": {
    "think": [
      { "provider": { "type": "open_ai", "model": "gpt-6-sol" }, "prompt": "You are concise." },
      { "provider": { "type": "open_ai", "model": "gpt-6-sol" } }
    ],
    "speak": [
      { "provider": { "type": "deepgram", "model": "aura-2-thalia-en" } },
      { "provider": { "type": "cartesia", "model_id": "sonic-3.6", "voice": { "mode": "id", "id": "skylar" } } }
    ]
  }
}

Every entry must be a model this service serves; one that is not fails Settings with INVALID_SETTINGS, as the primary does. When a provider fails you receive a THINK_REQUEST_FAILED or SPEAK_REQUEST_FAILED warning, and the failed provider is skipped until it answers again. Each turn is billed at the rate of the model that served it. A fallback that cannot be started when the session opens is skipped with a THINK_FALLBACK_UNAVAILABLE or SPEAK_FALLBACK_UNAVAILABLE warning. A speech-to-speech model has no separate stage to fall back from: it cannot be a fallback, and a speech-to-speech primary ignores its fallback entries (THINK_FALLBACK_IGNORED / SPEAK_FALLBACK_IGNORED).

Choosing models

Call GET /api/v1/voice/models for the models you can use, each with its measured latency. Settings accepts any id in that response except the managed deepgram-voice-* language models, which run only on Voice Agent (WebRTC) sessions; naming one here fails Settings with INVALID_SETTINGS. Filter on eligible to pick models measured fast enough to hold a conversation. Any speech-to-text × language model × text-to-speech combination is valid.

Eligibility is based on measured performance from real traffic, not on vendor claims. A model that is too slow to hold a conversation is marked ineligible and listed with the reason, so you can see why rather than wondering where it went.

Two speech-to-text models are streaming-capable here that you cannot use for file transcription in the same way:

  • ink-2 ($0.5625 / hr) — Cartesia's top-ranked voice-agent STT: 8% WER on AppTek's 14-accent call-centre benchmark, vs 10% Deepgram Flux and 12% ElevenLabs. It self-detects turns. English only — set "language": "en". Sending non-English audio does not error; it just transcribes badly. This is the only surface ink-2 runs on: the file transcription endpoint rejects it with a 400.
  • ink-whisper ($0.1875 / hr) — 100 languages including Hindi, Urdu and Tamil. Cartesia's cheapest STT, with dynamic chunking that reduces hallucination across pauses. Use this instead of ink-2 for any non-English call.

List available models

curl https://api.callmissed.com/api/v1/voice/models \
  -H "Authorization: Bearer cm_your_api_key"
{
  "turn_budget_ms": 1000,
  "llm": [
    {
      "id": "gpt-oss-120b",
      "label": "GPT-OSS 120B",
      "eligible": true,
      "verdict": "eligible",
      "p50_ms": 180.0,
      "samples": 240,
      "budget_ms": 400,
      "reason": "p50 180ms within the 400ms budget",
      "languages": ["en"],
      "voices": []
    }
  ],
  "stt": [],
  "tts": []
}
verdictMeaning
eligibleMeasured within its stage budget. Selectable.
too_slowMeasured over budget. Not offered; reason says by how much.
unsupportedStructurally unavailable (e.g. under maintenance).
unmeasuredNot enough measured turns yet to give a verdict.

p50_ms is null when a model has no measurement. It is never reported as 0 — zero would read as "instant" for a stage that was simply never measured.

FieldMeaning
turn_budget_msThe voice-to-voice target the three stage budgets add up to
id / labelModel id to send, and its display name
eligibletrue when verdict is eligible — the field to filter on
p50_ms / samplesMedian measured latency for the model's stage over the last 30 days, and the number of measured turns behind it
budget_msThe stage budget the model is judged against
reasonWhy the model got its verdict
languagesBCP-47 codes where recorded; [] means not recorded
voicesSelectable speakers (TTS models only; [] otherwise)

Server events

EventFieldsMeaning
Welcomerequest_idSocket open.
SettingsApplied—Configuration accepted; start streaming.
ConversationTextrole (user / assistant), contentA finished turn.
Historyrole and content, or function_calls[]A finished turn or completed function call, in agent.context.messages shape. Only with flags.history on.
UserStartedSpeaking—Barge-in. Stop playback and clear your buffer.
AgentThinkingcontentThe model is working.
AgentStartedSpeakingtotal_latency, tts_latency, ttt_latencyFirst audio of a turn. Seconds.
AgentAudioDone—Last audio chunk sent for this turn.
FunctionCallRequestfunctions[]Run a tool and reply.
FunctionCallCancelledfunctions[] (id, name)A request you already received was abandoned. Stop the work and do not reply.
LatencyReporteou_latency, stt_latency, ttt_latency, tts_latency, total_latency (on /v1/agent/converse: stt_latency, ttt_token_latency, tts_latency, total_latency)Per-stage timings for the turn. Seconds.
PromptUpdated / ThinkUpdated / ListenUpdated / SpeakUpdated—The matching update was applied.
InjectionRefused—An InjectAgentMessage was refused (see below).
Warningcode, descriptionNon-fatal; the session continues.
Errorcode, descriptionFatal; the socket closes. Reconnect to start a new session.

UserStartedSpeaking is the only barge-in signal — there is no separate flush message. When you receive it, stop playback and discard whatever audio you have buffered. If you don't, the caller keeps hearing the interrupted sentence for as long as your playback buffer is deep, which is the most common reason interruption appears not to work.

AgentAudioDone means the last chunk was sent, not that the caller heard it. Your own playback buffer may still be draining.

An interrupted reply's assistant ConversationText holds only the part the caller heard. If the caller barges in within the first fraction of a second of a reply, before any of it was heard, no assistant ConversationText is sent for that turn.

Latency fields are omitted when a stage was not measured, rather than reported as 0 (except AgentStartedSpeaking on /v1/agent/converse, see above).

Error codes

Where Deepgram has its own name for a code, /v1/agent/converse sends Deepgram's name; the native endpoint keeps its own.

Native codeOn /v1/agent/converseCause
SETTINGS_TIMEOUTCLIENT_MESSAGE_TIMEOUTNo Settings within 15 seconds of Welcome
NON_SETTINGS_MESSAGE_BEFORE_SETTINGSsameThe first message was not Settings
INVALID_JSONUNPARSABLE_CLIENT_MESSAGESettings was not valid JSON, or not a JSON object
INVALID_SETTINGSsameA field failed validation, a model id cannot be served, or Settings is too large
—SETTINGS_ALREADY_APPLIEDA second Settings arrived
CONCURRENCY_LIMITsameYour plan's browser/API voice sessions at once limit was reached while the session started
MODEL_UNAVAILABLEsameNo voice engine could be built for this configuration
INSUFFICIENT_CREDITSsameYour balance ran out mid-session
API_KEY_BUDGET_EXCEEDEDsameThe API key reached its spend limit, or was revoked or expired, mid-session
SESSION_TIMEOUTMAXIMUM_SESSION_LENGTH_REACHEDThe 2-hour session cap was reached
IDLE_TIMEOUTCLIENT_MESSAGE_TIMEOUTNothing was received from your client for 2 minutes (see Limits)
NO_INPUTsameThe caller stayed silent through every prompt (see Caller silence)
AGENT_ERRORsameThe voice engine could not continue
INTERNAL_ERRORINTERNAL_SERVER_ERRORThe voice engine could not start

Common Warning codes: INVALID_UPDATE, THINK_UPDATE_FAILED, LISTEN_UPDATE_FAILED, SPEAK_UPDATE_FAILED, FUNCTION_CALL_TIMEOUT, UNMATCHED_FUNCTION_RESPONSE, FUNCTION_ENDPOINT_UNSUPPORTED, UNKNOWN_MESSAGE, DUPLICATE_SETTINGS (native only), INVALID_JSON (UNPARSABLE_CLIENT_MESSAGE on converse), MESSAGE_TOO_LARGE, AUDIO_CHUNK_TOO_LARGE (the frame is dropped), PROMPT_TOO_LONG, STACK_SUBSTITUTED, CONTEXT_TRUNCATED, IDLE_TIMEOUT_APPROACHING, THINK_REQUEST_FAILED, SPEAK_REQUEST_FAILED, THINK_FALLBACK_UNAVAILABLE, SPEAK_FALLBACK_UNAVAILABLE, THINK_FALLBACKS_TRUNCATED, SPEAK_FALLBACKS_TRUNCATED, THINK_FALLBACK_IGNORED, SPEAK_FALLBACK_IGNORED, INTERNAL_ERROR (INTERNAL_SERVER_ERROR on converse: one message could not be applied, the session continues), and, on /v1/agent/converse only, MAXIMUM_SESSION_LENGTH_APPROACHING five minutes before the 2-hour cap.

Tool calling

Declare tools in Settings (agent.tools on the native surface, agent.think.functions on the Deepgram-compatible one), then answer requests over the same socket.

{
  "type": "FunctionCallRequest",
  "functions": [
    {
      "id": "fc_01H...",
      "name": "lookup_order",
      "arguments": "{\"order_id\":\"A-1042\"}",
      "client_side": true
    }
  ]
}

Reply with the result. arguments is a JSON string, not an object. Echo the id; content should be a string (anything else is JSON-encoded for you).

{
  "type": "FunctionCallResponse",
  "id": "fc_01H...",
  "name": "lookup_order",
  "content": "{\"status\":\"shipped\",\"eta\":\"2 days\"}"
}

A tool that does not answer within 30 seconds does not hang the call — the agent tells the caller it could not complete the action, and a FUNCTION_CALL_TIMEOUT warning is emitted.

Server-side execution (a functions[].endpoint) is not supported. Declaring one returns a Warning and the call is dispatched to your client instead, so an unsupported mode is never silently ignored.

If the agent abandons a request it already sent you, you receive FunctionCallCancelled with that request's id and name. Stop the work and do not reply; a late FunctionCallResponse for it is dropped.

Updating a live session

MessageBodyEffectAcknowledged by
UpdatePromptpromptNative: replaces the system prompt. /v1/agent/converse: adds to it, as Deepgram does. The result is capped at 25,000 characters (PROMPT_TOO_LONG)PromptUpdated
UpdateThinkmodel, optional temperature, prompt, functionsSwitch the language model (and optionally the prompt). functions present replaces the tool set ([] removes every tool); absent keeps itThinkUpdated
UpdateListenmodel, optional languageSwitch speech recognitionListenUpdated
UpdateSpeakmodel, voice, optional languageSwitch voice or text-to-speech modelSpeakUpdated
InjectAgentMessagemessage, optional behaviorMake the agent say something nowInjectionRefused if refused
InjectUserMessagecontentInject text as if the caller said it—
ForceEndTurn—End the caller's turn now (push-to-talk release, a keypad press, your own end-of-speech detection); the agent replies. Ignored when the caller is not mid-turn—
KeepAlive—Hold an idle socket open (does not extend the 2-hour cap)—
{ "type": "UpdateSpeak", "model": "sonic-3.6", "voice": "skylar" }

The update bodies can be sent flat as above, or nested the Deepgram way ({"type": "UpdateSpeak", "speak": {"provider": {"model": "sonic-3.6"}}}) on either surface.

InjectAgentMessage behavior is default, queue or interrupt. default is refused while either side is speaking, queue only while the caller is speaking, and interrupt is never refused: it cuts off whatever the agent is saying and speaks the new message instead. queue speaks after the current reply.

An UpdateThink fallback array applies its first entry only (THINK_FALLBACK_IGNORED); a think.endpoint in it is reported with SETTINGS_FIELDS_IGNORED.

A model id the service cannot serve keeps the current model and returns a Warning; it is never swapped for a different one behind your back. A model, voice or language that is not a string returns an INVALID_UPDATE warning and changes nothing.

Latency

The fast path targets sub-1s from end of your speech to first audio back. That budget is the sum of three serial stages:

StageBudget
Speech recognition, final transcript300 ms
Language model, first token400 ms
Text-to-speech, first byte300 ms

LatencyReport gives you the real numbers per turn, so you can measure rather than take our word for it.

Sub-200ms end-to-end voice-to-voice is not achievable with a speech-to-text → language model → speech pipeline, by anyone. Detecting that you stopped speaking alone costs more than that. Treat sub-second as the realistic target and measure the rest with LatencyReport.

Limits

LimitValue
Maximum session length2 hours
Time to send Settings15 seconds
Tool response timeout30 seconds
Prompt length25,000 characters
Audio in one binary frame5 seconds at your input sample rate
Idle time with nothing from your client2 minutes (warning after 90 seconds)
agent.context.messages100 entries, 60,000 characters
Fallback entries per array3 after the primary
Browser/API voice sessions at oncePer plan (Free 1, Starter 5, Pro 20, Enterprise and above unlimited)
Models named in SettingsMust be inside the key's allowed_models list

A model you name in Settings that the key's allowed_models list excludes fails Settings with INVALID_SETTINGS ("not allowed for this API key"); the same model named in a live update is refused with an INVALID_UPDATE warning and the session keeps the model it was running. Models you leave out are not checked.

JSON control messages are small; one far larger than any legitimate update is dropped with a MESSAGE_TOO_LARGE warning. A Settings message whose prompt is within the 25,000-character limit always fits.

This is a temporary "at capacity, retry" response, not a plan limit. When the service as a whole is briefly at capacity, the WebSocket upgrade is refused before it opens, with HTTP 503, a Retry-After header (seconds) and a JSON body {"error": {"code": "SERVER_AT_CAPACITY", "message": "..."}}. Clients or proxies that cannot read an upgrade response see a failed handshake; reconnect with backoff. Sessions already running are not affected.

Connection attempts are also admission-controlled before the API key is checked. Opening many sessions at once, or reconnecting in a tight loop, from one server can be refused at the handshake before your plan's concurrent-session cap is reached. Reconnect with backoff, and contact [email protected] if you need sustained high concurrency from one server.

A session whose client sends nothing (no audio, no KeepAlive, no message) for 2 minutes is closed with an IDLE_TIMEOUT error (CLIENT_MESSAGE_TIMEOUT on /v1/agent/converse), after an IDLE_TIMEOUT_APPROACHING warning 30 seconds before. Time the agent spends replying, or waiting on one of your function calls, does not count. If you pause your audio stream, send KeepAlive every few seconds. Otherwise a session keeps running, and keeps transcribing whatever audio you send, until you close the socket or it reaches the 2-hour cap, so close it when the conversation ends.

Caller silence

Off by default. Set agent.nudge_max (1–5) in Settings and, when nobody has spoken for agent.nudge_delay_seconds, the agent says agent.nudge_message. The count starts over whenever the caller speaks. After the last unanswered prompt the session ends with a NO_INPUT error, unless agent.nudge_hangup is false. These are the same settings, with the same bounds, as a saved agent's no-input prompts; they work on both endpoints.

Usage is billed per turn across speech recognition, the language model and text-to-speech, at each model's own rate. The session also pays a flat ₹1 a minute platform fee, charged once when it ends, per second above a 30-second minimum. If your balance runs out mid-session you receive an INSUFFICIENT_CREDITS Error frame and the socket closes, rather than the call continuing unbilled. An API key with a spend limit is checked after every billed turn as well; once the key reaches its limit you receive API_KEY_BUDGET_EXCEEDED and the socket closes.

Not available on this endpoint

A session here is configured entirely by its Settings message; there is no bot_id. Features loaded from a saved agent are not applied: knowledge base, the built-in agent tools, pronunciations, {{variables}}, PII redaction, background sound and noise suppression (the no-input prompts are set in Settings, see Caller silence). For those, create the session with the Voice Session API and a bot_id.

On this page