> ## Documentation Index
> Fetch the complete documentation index at: https://rimelabs-docs-coda-websocket-reference.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Coda WebSocket API

> Connection setup, context lifecycle, message fields, audio formats, and errors for Coda WebSocket v1.

Connect to `wss://api.rime.ai/coda/ws` to send text and receive audio on one persistent connection. Each synthesis has a `contextId`. JSON and Protobuf support the same parameters, audio formats, and lifecycle. Neither returns word timestamps.

For runnable clients, use the [JSON quickstart](/api-reference/coda/websockets-json), [Protobuf quickstart](/api-reference/coda/websockets-binary), or [LiveKit integration](/api-reference/coda/websockets-livekit).

## Connect and authenticate

Your server sends `Authorization: Bearer YOUR_API_KEY` in the WebSocket upgrade request. Create a key on the [API Tokens page](https://app.rime.ai/tokens). Keep it on your server. Browser applications must connect through your backend.

Select one subprotocol in the upgrade request and check the negotiated value:

| `Sec-WebSocket-Protocol` | Server response format                                    |
| ------------------------ | --------------------------------------------------------- |
| `rime.v1.json`           | JSON text messages; base64 audio                          |
| `rime.v1.binary`         | Protobuf binary messages; audio bytes inside the envelope |
| No subprotocol           | JSON text messages                                        |

Wait for `ready` and check that `ready.protocol` is `1` before sending `start`. `ready.languages` lists supported language tags. `ready.defaultLanguage`, when present, gives the deployment default. Readiness reports engine availability; it does not replace authentication.

## Client messages

Each message contains one payload field and an optional `contextId`. Do not send a `type` discriminator or a `payload` wrapper. JSON uses camelCase names; generated Python Protobuf objects use snake\_case, such as `audio_parameters.sampling_rate`.

| Payload  | Type             | Purpose                                                                |
| -------- | ---------------- | ---------------------------------------------------------------------- |
| `config` | Object           | Set connection defaults or credentials once, before the first `start`. |
| `start`  | Synthesis object | Open a context with complete or streaming text input.                  |
| `text`   | String           | Append a complete sentence or stable clause to streaming input.        |
| `end`    | Empty object     | Finish streaming input. Audio continues until `done`.                  |
| `cancel` | Empty object     | Cancel a context and receive `cancelled`.                              |

Use the same `contextId` for `start`, `text`, `end`, and `cancel` in one turn. A repeated `end`, or one for a complete-text or unknown context, has no effect.

This message appends a sentence to an open streaming context:

```json theme={null}
{
  "contextId": "turn-1",
  "text": "Hello. How can I help you? "
}
```

## Run a synthesis context

The value of `start.text` selects the input mode:

| Input mode     | `start.text`                       | Later input          | Completion                           |
| -------------- | ---------------------------------- | -------------------- | ------------------------------------ |
| Complete text  | Contains all text                  | Do not append `text` | Read until `done`; no `end` required |
| Streaming text | Omitted, empty, or whitespace only | Send `text` messages | Send `end`, then read until `done`   |

This sequence streams two sentences. The client can send text while it receives audio:

```mermaid theme={null}
sequenceDiagram
    participant Client
    participant Coda
    Client->>Coda: Upgrade with authorization and subprotocol
    Coda-->>Client: ready
    Client->>Coda: start with empty text and contextId
    Coda-->>Client: started with requestId
    Client->>Coda: text, complete sentence
    Coda-->>Client: audio
    Client->>Coda: text, next sentence
    Coda-->>Client: audio
    Client->>Coda: end
    Coda-->>Client: remaining audio
    Coda-->>Client: done
    Note over Client,Coda: Connection remains open for the next context
```

### Complete text in one request

This `start` supplies the complete utterance. The server finishes it without an `end` message:

```json theme={null}
{
  "contextId": "turn-1",
  "start": {
    "speaker": "lyra",
    "language": "en",
    "text": "Hello. How can I help you?"
  }
}
```

### Stream text as it becomes available

* Buffer LLM tokens locally, then send complete sentences or stable clauses. The server normalizes each message separately, so splitting a number, abbreviation, or word can change its pronunciation.
* Preserve spaces at chunk boundaries, for example, `"Hello. "` followed by `"How can I help? "`.
* An input pause needs no message. Send `end` only when the turn is complete. It leaves the socket open.

[`splitStrategy`](#text-splitting-and-lookahead) splits text within each message. It does not buffer incomplete sentences across messages.

### Identify and reuse contexts

* Use a new `contextId` per turn to help reject stale audio. An omitted or empty ID selects `default`, which the server returns as `"contextId":"default"`.
* Responses from concurrent contexts can arrive between each other. Route them by `contextId` from one receive loop, with a separate decoder and playback queue per context.
* You can reuse an ID after its terminal event. The engine supports [up to 16 open contexts](#limits-and-connection-health).
* Log `started.requestId` with your `contextId` for error reports. A request can fail or be cancelled before `started`, so handle terminal events as soon as you send `start`.

### Stop speech on interruption

1. Stop local playback and discard the turn's queued audio.
2. Send `cancel` with that turn's `contextId`.
3. Keep reading until `cancelled`, or `done` if synthesis completed first. A context-scoped `error` also ends the run. Discard audio for the interrupted turn that was already in transit.

This message cancels only `turn-1`:

```json theme={null}
{
  "contextId": "turn-1",
  "cancel": {}
}
```

Cancellation is idempotent. An unknown or finished context still gets a `cancelled` acknowledgement. Cancelling a context leaves the connection open; closing the socket cancels all its active contexts.

## Connection configuration

Send at most one `config` message before the first `start`. It has no acknowledgement.

| `config` field  | Type             | Behavior                                                                                     |
| --------------- | ---------------- | -------------------------------------------------------------------------------------------- |
| `defaults`      | Synthesis object | Defaults for later `start` messages. Its `text` field is ignored.                            |
| `authorization` | String           | Full authorization value. See [message-based authentication](#message-based-authentication). |
| `license`       | String           | License JSON as a string, for deployments that use license authentication.                   |

Defaults merge at the top level. `start.audioParameters` replaces the whole default object, so include every audio setting you want to retain. The same rule applies to `codaParameters`.

This configuration uses header authentication and sets PCM output for later turns:

```json theme={null}
{
  "config": {
    "defaults": {
      "speaker": "lyra",
      "language": "en",
      "audioParameters": {
        "audioFormat": "audio/pcm",
        "samplingRate": 24000
      }
    }
  }
}
```

If a later `start.audioParameters` contains only `{"samplingRate":8000}`, it discards the default `audioFormat`. Include `"audioFormat":"audio/pcm"` to retain PCM.

## Synthesis parameters

Set these fields in `start` or `config.defaults`. The URL selects Coda; there is no `modelId` field.

| Field             | Type   | Default             | Use                                                                              |
| ----------------- | ------ | ------------------- | -------------------------------------------------------------------------------- |
| `speaker`         | String | Deployment-specific | Set a [Coda voice](/docs/voices-coda), such as `lyra`, explicitly.               |
| `language`        | String | Deployment default  | BCP 47 tag, such as `en`. Supported tags appear in `ready.languages`.            |
| `text`            | String | Empty               | Select the [input mode](#run-a-synthesis-context). Ignored in `config.defaults`. |
| `audioParameters` | Object | See below           | Audio format, sampling rate, and speed.                                          |
| `splitStrategy`   | Enum   | Deployment default  | Split each input message. See [values below](#text-splitting-and-lookahead).     |
| `codaParameters`  | Object | See below           | Streaming text lookahead.                                                        |

Coda does not accept `temperature`, `topP`, `topK`, `repetitionPenalty`, `maxTokens`, or `seed`. Unknown fields can be ignored, so a successful request does not prove an unsupported setting took effect.

### Audio parameters

| `audioParameters` field | Type    | Default     | Behavior                                                                                   |
| ----------------------- | ------- | ----------- | ------------------------------------------------------------------------------------------ |
| `audioFormat`           | String  | `audio/wav` | Use a MIME type from the table below.                                                      |
| `samplingRate`          | Integer | `24000` Hz  | Positive rate supported by your decoder or player. Set `8000` for 8 kHz telephony.         |
| `timeScaleFactor`       | Number  | `1.0`       | Below `1` is faster; above `1` is slower. The engine clamps values to `0.4` through `2.5`. |

Both protocols support these audio formats. Decode the message envelope first: JSON carries base64 audio; Protobuf carries audio bytes.

| MIME type                | Encoded content                           | Client handling                                                                       |
| ------------------------ | ----------------------------------------- | ------------------------------------------------------------------------------------- |
| `audio/pcm`              | Raw mono, signed 16-bit little-endian PCM | Use the requested sampling rate. There is no file header.                             |
| `audio/wav`              | WAV with 16-bit PCM                       | Feed chunks to one streaming WAV decoder.                                             |
| `audio/mpeg`             | MP3                                       | Feed chunks to one MP3 decoder, or append them to a file.                             |
| `audio/ogg;codecs=opus`  | Opus in Ogg                               | Use an Ogg/Opus decoder.                                                              |
| `audio/webm;codecs=opus` | Opus in WebM                              | Use a WebM/Opus decoder.                                                              |
| `audio/pcmu`             | G.711 mu-law                              | Decode one byte per sample at the requested rate. Set `samplingRate: 8000` for 8 kHz. |

Message boundaries are transport chunks, not separate files or guaranteed decoder frames. Keep one decoder alive for each context. Raw PCM consumers must retain any incomplete sample between chunks.

<Accordion title="Direct engine audio aliases">
  Direct engine requests also accept `audio/mp3`, `audio/ogg`, `audio/webm`, `audio/x-mulaw`, and `audio/l16`. The `audio/l16` alias returns little-endian PCM. Use `audio/pcm` to avoid confusion with network-byte-order L16. Unknown formats fall back to WAV; an empty format is invalid. LiveKit accepts only the six canonical MIME types above.
</Accordion>

### Text splitting and lookahead

| `splitStrategy` value                   | Behavior                                                         |
| --------------------------------------- | ---------------------------------------------------------------- |
| `SPLIT_STRATEGY_NONE`                   | Keep each input message intact.                                  |
| `SPLIT_STRATEGY_SENTENCE`               | Split each message into sentences.                               |
| Omitted or `SPLIT_STRATEGY_UNSPECIFIED` | Use the deployment default, `NONE` in the standard Coda package. |

Set `codaParameters.textLookaheadTokens` to a non-negative integer, default `0`. More leading text tokens can improve the start of an utterance but delay first audio. The engine caps lookahead at the available tokens and ignores it for complete-text input.

## Server events

| Event       | Scope                 | Meaning                                                                     |
| ----------- | --------------------- | --------------------------------------------------------------------------- |
| `ready`     | Connection            | Engine ready. Check protocol version and supported languages.               |
| `started`   | Context               | Synthesis accepted. Save `requestId` for error reports.                     |
| `audio`     | Context               | Audio chunk: base64 in JSON, bytes in Protobuf.                             |
| `done`      | Context               | All audio delivered. Finish decoding; playback can still have queued audio. |
| `cancelled` | Context               | Context cancelled. Discard remaining audio and release its state.           |
| `error`     | Context or connection | See [completion and errors](#handle-completion-and-errors).                 |

These examples show separate server messages. The language list is illustrative, and `error.requestId` is optional:

<CodeGroup>
  ```json ready theme={null}
  {
    "ready": {
      "protocol": 1,
      "languages": ["en"],
      "defaultLanguage": "en"
    }
  }
  ```

  ```json started theme={null}
  {
    "contextId": "turn-1",
    "started": {"requestId": "request-123"}
  }
  ```

  ```json audio theme={null}
  {
    "contextId": "turn-1",
    "audio": "AQID"
  }
  ```

  ```json done theme={null}
  {
    "contextId": "turn-1",
    "done": {}
  }
  ```

  ```json cancelled theme={null}
  {
    "contextId": "turn-1",
    "cancelled": {}
  }
  ```

  ```json error theme={null}
  {
    "contextId": "turn-1",
    "error": {
      "kind": "invalid_input",
      "message": "Invalid request",
      "requestId": "request-123"
    }
  }
  ```
</CodeGroup>

Proto3 JSON omits default-valued fields. Test payload presence with `"done" in event`; the empty object `{}` is false in Python. For Python Protobuf, use `response.WhichOneof("payload")`. Allow unknown fields for compatible schema additions.

### Handle completion and errors

An active context finishes with one terminal event: `done`, `cancelled`, or a context-scoped `error`. No more audio belongs to that run after its terminal event.

| Failure                                      | Client action                                                                                      |
| -------------------------------------------- | -------------------------------------------------------------------------------------------------- |
| `error` with a non-empty `contextId`         | Stop that context. Log `kind`, `message`, and any `requestId`.                                     |
| `error` with no `contextId`, or an empty one | Handle a connection-level error. Existing contexts can remain active.                              |
| Socket closes before a terminal event        | Treat unfinished contexts as interrupted. Reconnect and wait for `ready` before starting new work. |

Duplicate `start`, text outside an open streaming input, and too many open contexts cause connection-level errors. They do not terminate an existing context. Malformed envelopes and failed authentication can close the socket.

There is no resume operation. Retrying text whose audio already played can repeat speech. Choose retries at a turn or sentence boundary.

### Error kinds

`error.kind` is a string, not an HTTP status code. Authentication can also fail with HTTP `401` during the upgrade.

| Kind                 | Action                                                            |
| -------------------- | ----------------------------------------------------------------- |
| `invalid_input`      | Correct fields, text, or message order before retrying.           |
| `unauthenticated`    | Check the API key or deployment credentials.                      |
| `permission_denied`  | Check access to the requested service.                            |
| `not_found`          | Check the requested resource, such as the voice.                  |
| `resource_exhausted` | Reduce concurrency and retry with backoff.                        |
| `unavailable`        | Retry with backoff if the application can repeat the turn safely. |
| `timeout`            | Check timeouts and service health before retrying.                |
| `unimplemented`      | Use a supported operation or configuration.                       |
| `internal`           | Record the request ID and report the failure.                     |

An undecodable envelope produces `invalid_input` and close code `1007`. Invalid or conflicting message credentials produce close code `1008`. Treat an unexpected close before a terminal event as incomplete synthesis, even if some audio arrived.

## Limits and connection health

| Engine limit                    | Value                             |
| ------------------------------- | --------------------------------- |
| Incoming message or frame       | 64 KiB, including the envelope    |
| Open contexts per connection    | 16                                |
| Context ID                      | 128 UTF-8 bytes                   |
| Authentication through `config` | Within 10 seconds                 |
| Server ping interval            | 30 seconds                        |
| Read inactivity timeout         | 75 seconds; client pongs reset it |

A hosted gateway can apply additional limits. Use a WebSocket library that answers ping frames. Keep one reader active, await sends to respect backpressure, and bound your audio queue to limit memory use with a slow player.

## Advanced connection options

### Subprotocol negotiation

If you offer both supported subprotocols, the server selects the first in your list. An offer with only unknown values gets no selected subprotocol, which can cause the client to reject the handshake.

The server decodes incoming messages by their WebSocket type, text or binary. The selected subprotocol fixes the response encoding for the connection. Use the same encoding in both directions unless you need a mixed client.

### Message-based authentication

Clients that cannot send upgrade headers can send `config.authorization`, such as `Bearer YOUR_API_KEY`, in their first message. This does not make an API key safe to put in a browser.

On a direct engine connection, send credentials within 10 seconds. Do not send credentials in both the upgrade headers and `config`; the engine closes the connection. Deployments with license authentication accept `config.license` as a JSON string.