Managed Voice Agent

Overview

The Managed Voice Agent is a full speech-to-speech pipeline behind a single WebSocket. You stream microphone audio in; you get synthesized speech and conversation events back. Speech recognition, the language model, text-to-speech, turn-taking and interruption handling are all run and tuned for you.

Unlike the Voice Session API, there is no WebRTC and no client SDK to install — a plain WebSocket and raw PCM is the whole integration.

Two wire protocols are served, backed by the same engine:

If you already have an integration written against Deepgram's Voice Agent API, point it at the second URL and it will work unchanged.

Both live on api.callmissed.com, the same host as the rest of the API — there is no separate hostname to allowlist.

Authentication

Both endpoints take an API key with stt, tts and llm permissions.

Authorization: Token cm_your_api_key

Browsers cannot set headers on a WebSocket, so the subprotocol form is also accepted:

new WebSocket(url, ["token", "cm_your_api_key"])

Plan limits, the concurrent-session cap and your credit balance are all checked before the socket is accepted, so an over-limit connection fails at the handshake rather than mid-conversation.

Session flow

1. Connect. The server sends Welcome with a request_id.
2. Send Settings as your first message.
3. Wait for SettingsApplied, then start streaming audio.
4. Send raw PCM as binary frames; receive synthesized PCM as binary frames and events as JSON on the same socket.

{ "type": "Welcome", "request_id": "fc553ec9-5874-49ca-a47c-b670d525a4b1" }

Settings (native)

The native shape is organised around what you actually choose: one model id per role.

{
  "type": "Settings",
  "audio": {
    "input":  { "encoding": "linear16", "sample_rate": 24000 },
    "output": { "encoding": "linear16", "sample_rate": 24000 }
  },
  "agent": {
    "prompt": "You are a concise support agent for an Indian retail brand.",
    "greeting": "Hi, how can I help?",
    "language": "en-IN",
    "llm": { "model": "gpt-oss-120b", "temperature": 0.4 },
    "stt": { "model": "saaras:v3" },
    "tts": { "model": "bulbul:v3", "voice": "shubh" }
  },
  "tags": ["support"]
}

Every other client message — updates, injection, tool responses, keepalive — is identical across both protocols. Switching protocols means rewriting one message.

Audio format

linear16 (raw 16-bit PCM, little-endian, mono) in both directions, at 8000, 16000, 24000, 32000 or 48000 Hz. Output is raw frames with no container: a WAV or OGG header would be read as audio by most telephony consumers.

An unsupported encoding, sample rate or container is rejected at Settings time rather than accepted and quietly changed, so you find out at the handshake instead of hearing the wrong thing on a call.

Choosing models

Call GET /api/v1/voice/models for the models you can use, each with its measured latency. Any eligible speech-to-text × language model × text-to-speech combination is valid.

Models are offered based on measured performance from real traffic, not on vendor claims. A model that is too slow to hold a conversation is not offered — it is listed with the reason, so you can see why rather than wondering where it went.

List available models

curl https://api.callmissed.com/api/v1/voice/models \
  -H "Authorization: Bearer cm_your_api_key"

{
  "turn_budget_ms": 1000,
  "llm": [
    {
      "id": "gpt-oss-120b",
      "label": "GPT-OSS 120B",
      "eligible": true,
      "verdict": "eligible",
      "p50_ms": 180.0,
      "samples": 240,
      "budget_ms": 400,
      "reason": "p50 180ms within the 400ms budget",
      "languages": ["en"],
      "voices": []
    }
  ],
  "stt": [],
  "tts": []
}

p50_ms is null when a model has no measurement. It is never reported as 0 — zero would read as "instant" for a stage that was simply never measured.

Server events

Latency fields are omitted when a stage was not measured, rather than reported as 0.

Tool calling

Declare tools in Settings, then answer requests over the same socket.

{
  "type": "FunctionCallRequest",
  "functions": [
    {
      "id": "fc_01H...",
      "name": "lookup_order",
      "arguments": "{\"order_id\":\"A-1042\"}",
      "client_side": true
    }
  ]
}

Reply with the result. arguments is a JSON string, not an object.

{
  "type": "FunctionCallResponse",
  "id": "fc_01H...",
  "name": "lookup_order",
  "content": "{\"status\":\"shipped\",\"eta\":\"2 days\"}"
}

A tool that does not answer within 30 seconds does not hang the call — the turn continues and a Warning is emitted.

Server-side execution (a functions[].endpoint) is not supported. Declaring one returns a Warning and the call is dispatched to your client instead, so an unsupported mode is never silently ignored.

Updating a live session

A model id the service cannot serve keeps the current model and returns a Warning; it is never swapped for a different one behind your back.

Latency

The fast path targets sub-1s from end of your speech to first audio back. That budget is the sum of three serial stages:

LatencyReport gives you the real numbers per turn, so you can measure rather than take our word for it.

Limits

Usage is billed per turn across speech recognition, the language model and text-to-speech. If your balance runs out mid-session you receive an Error frame and the socket closes, rather than the call continuing unbilled.