Stream

Use the VoxML Stream verb to connect an answered call to a WebSocket service. It is intended for real-time transcription, conversational AI, quality monitoring, and other applications that need access to the call audio.

TryVox begins the stream while executing the call's VoxML response. There is currently no separate REST API for creating, listing, retrieving, or stopping stream objects on an already-running call.

Basic example

{
  "voxml_version": "1.0",
  "instructions": [
    {
      "verb": "Stream",
      "url": "wss://voice.example.com/calls",
      "track": "inbound_track",
      "parameters": {
        "agent_id": "support-assistant",
        "language": "en-IN"
      }
    }
  ]
}

Fields

outbound_track is not currently supported. TryVox returns a stream execution error instead of silently substituting another track.

Choosing a track

Use inbound_track for speech recognition or an AI assistant that should listen to the caller:

{
  "verb": "Stream",
  "url": "wss://voice.example.com/listen",
  "track": "inbound_track"
}

Use both_tracks for analytics, compliance, or transcription that needs the complete conversation:

{
  "verb": "Stream",
  "url": "wss://voice.example.com/analyse",
  "track": "both_tracks"
}

With both_tracks, the two directions are mixed into one mono channel. They are not delivered as separately labelled tracks.

WebSocket connection

Your server must:

• Accept a WebSocket client connection from TryVox.
• Accept the audio.drachtio.org WebSocket subprotocol.
• Handle one initial UTF-8 text frame followed by binary audio frames.
• Keep reads non-blocking and process audio continuously.

The connection sequence is:

1. TryVox opens the WebSocket.
2. TryVox sends the parameters object as a JSON text frame.
3. TryVox sends raw PCM audio in binary frames.
4. The connection closes when the call channel or stream ends.

For example, these parameters:

{
  "parameters": {
    "account_ref": "acct_42",
    "conversation_ref": "conv_91"
  }
}

arrive as the first text frame:

{
  "account_ref": "acct_42",
  "conversation_ref": "conv_91"
}

Do not treat the first text frame as audio.

Audio format

Audio sent from TryVox to your server uses:

The audio frames are binary WebSocket messages. They are not JSON objects and are not Base64 encoded.

If your speech or model provider expects 16 kHz audio, resample the PCM stream after receiving it. Do not reinterpret 8 kHz bytes as 16 kHz audio.

Authentication

The url may contain a short-lived, call-scoped token:

{
  "verb": "Stream",
  "url": "wss://voice.example.com/calls?token=eyJhbGciOi...",
  "track": "inbound_track"
}

Prefer an expiring token over a permanent API key. Validate it during the WebSocket upgrade and bind it to the expected account or call. Avoid placing reusable account credentials in parameters, because metadata travels inside the established WebSocket rather than authenticating the initial upgrade request.

Custom WebSocket request headers are not currently configurable through the Stream verb.

Stream termination

The WebSocket close event is the stream-level termination signal. Release model sessions, buffers, and temporary resources when it arrives.

For the authoritative call result, also configure the call's hangup_url. A stream can close because of a call hangup, a network failure, or media-server shutdown, so the close event alone does not contain the final call disposition.

TryVox does not currently emit separate Stream status callbacks or expose a Stream ID.

Sending audio toward the call

The same WebSocket can send audio back to the caller. Send a UTF-8 JSON text frame with type: "playAudio":

{
  "type": "playAudio",
  "data": {
    "audioContentType": "raw",
    "sampleRate": 8000,
    "audioContent": "BASE64_ENCODED_PCM_AUDIO"
  }
}

audioContent must be Base64-encoded mono signed linear PCM. For raw audio, sampleRate must be 8000 or 16000. You may instead send a complete WAV file by setting audioContentType to wave and Base64-encoding the file.

Send whole utterances, not real-time frames.

Each message becomes its own playback on the call leg. Consecutive messages leave roughly a 100 ms seam, so streaming 20 ms frames the way you would to a media-streams API produces far more seam than audio and sounds stuttered and robotic. Measured on real calls, ~500 ms chunks produced 24 audible dropouts in 37 seconds, while ~3.4 s chunks were almost seamless.

Buffer your agent's speech and send it in as few messages as possible, in playback order. Send each message as soon as it is ready — do not pace it to real time. Messages are dispatched through a bounded internal queue, so a flood of small chunks can be dropped outright rather than merely seamed.

Interrupting the agent (barge-in)

When the caller starts speaking over the agent, send a killAudio text frame:

{ "type": "killAudio" }

This stops audio that is currently playing and discards anything you have already sent that has not started yet. Without it, queued audio would begin playing the moment the current chunk was cut short and the caller would still be talked over.

Detecting the interruption is your side's job — you have the caller's audio in real time, so run your own voice-activity or turn detection and send killAudio as soon as you decide the caller has the floor.

Because sent audio can be cancelled this way, it is safe to send a reply as soon as it is ready rather than waiting to be certain the caller has finished.

checkpoint and stop are not defined. Use the WebSocket close event for stream cleanup and the call API or VoxML for call control.

AI assistant example

{
  "voxml_version": "1.0",
  "instructions": [
    {
      "verb": "Say",
      "text": "Please wait while I connect the assistant."
    },
    {
      "verb": "Stream",
      "url": "wss://assistant.example.com/media?session=short-lived-token",
      "track": "inbound_track",
      "parameters": {
        "assistant": "customer-care",
        "locale": "en-IN"
      }
    }
  ]
}

Operational guidance

• Use wss:// with a publicly trusted certificate in production.
• Authorize the WebSocket during the upgrade, before accepting audio.
• Expect the first frame to be text and all subsequent audio frames to be binary.
• Apply backpressure and keep per-frame processing short.
• Resample explicitly when downstream services require another sample rate.
• Treat WebSocket disconnects as normal cleanup events and use hangup_url for the final call status.
• Do not log access tokens or raw call audio unless your retention and consent policies allow it.

Current limitations

• Streams can only be started through VoxML.
• The input format is fixed to 8 kHz L16.
• Only inbound_track and mono-mixed both_tracks are supported.
• Stream IDs, REST lifecycle operations, reconnect controls, status callbacks, and custom upgrade headers are not exposed.
• Bidirectional playback supports playAudio; checkpoint, clear, and stop control events are not exposed.