Realtime Audio

​

Overview

• Same auth surface as the rest of Mesh — your rsk_... data-plane token.
• For speech-to-speech models the wire format is identical to OpenAI’s Realtime API. Mesh passes JSON event bodies through verbatim in both directions, so any client written against the upstream spec works against Mesh by switching the WebSocket URL. elevenlabs/scribe_v2_realtime speaks ElevenLabs’ Scribe frames instead. See Speech to Text.
• Billed on usage, settled every 60 seconds during the session and at close.

​

Quickstart

// Browsers can't set request headers on a WebSocket, so pass the key
// via the api_key query parameter.
const ws = new WebSocket(
  `wss://api.meshapi.ai/v1/realtime?model=openai/gpt-realtime-2&api_key=${MESH_API_KEY}`,
);

ws.addEventListener("open", () => {
  ws.send(JSON.stringify({
    type: "session.update",
    session: {
      type: "realtime",
      output_modalities: ["audio"],
      audio: {
        output: { format: { type: "audio/pcm", rate: 24000 }, voice: "alloy" },
      },
    },
  }));
});

ws.addEventListener("message", (e) => {
  const event = JSON.parse(e.data);
  if (event.type === "response.output_audio.delta") {
    playAudio(event.delta);
  }
});

# Use websocat (or wscat) for a quick connectivity check. The key goes in
# the subprotocol or ?api_key=; an Authorization header is not read.
websocat \
  "wss://api.meshapi.ai/v1/realtime?model=openai/gpt-realtime-2&api_key=$MESH_API_KEY"

​

Authentication

Sec-WebSocket-Protocol: openai-realtime, Bearer <YOUR_RSK_KEY>

wss://api.meshapi.ai/v1/realtime?model=openai/gpt-realtime-2&api_key=rsk_...

​

Message protocol

​

Supported models

​

Pricing

​

Pre-flight pricing

​

Billing

• Balance hold. Opening a session holds up to $10 of your balance until it closes (less for models priced by duration or sessions with a token cap; no hold on BYOK). If your balance can’t cover the hold, the session is refused with insufficient_quota. If your account balance is exhausted during a session, the connection is closed with an insufficient_quota error. Top up to reconnect. Partial responses up to the point of disconnect are still billed.
• Concurrent sessions. An account can hold 5 concurrent realtime sessions by default (a model can set its own cap in realtime_max_concurrent_per_owner on GET /v1/models). The count is shared with streaming TTS and STT sessions.
• Session token caps. Sessions configured with a max-token cap close with a session_token_cap_exceeded error once the cap is reached.

​

Limits and known caveats

• Session length. Mesh closes a session after 30 minutes with session_duration_exceeded. Long-running agents should reconnect and resume application-level state.
• Idle timeout. Mesh closes a socket that receives no client→server frame for 60 seconds, with idle_timeout. Send a session.update ping or keep the audio buffer flowing.
• Browsers / Safari. Browsers can’t set request headers on a WebSocket and can’t reliably send the space-separated Bearer <key> subprotocol token. In the browser, authenticate with the ?api_key=<rsk_...> query fallback.
• No HTTP fallback. This endpoint only exists as a WebSocket upgrade; a plain HTTP request to /v1/realtime is not served.

​

Errors

​

error.code reference

​

WebSocket close codes (incidental)

​

Next steps

• For the ElevenLabs Scribe realtime STT frame protocol, see Speech to Text → WebSocket Real-Time STT.
• Symptom-by-symptom fixes live in Troubleshooting → Realtime Audio.
• Review the Authentication guide for key rotation and scoping.
• See the API reference entry for the OpenAPI stub.
• Watch the upstream OpenAI Realtime API reference for new event types — Mesh forwards them without code changes.