> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://developers.meshapi.ai/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://developers.meshapi.ai/_mcp/server.

# Realtime Audio

> Bidirectional speech-to-speech WebSocket sessions with the Node.js SDK.

# Realtime Audio

`client.realtime` opens a bidirectional WebSocket session to `wss://api.meshapi.ai/v1/realtime`. The wire format is identical to OpenAI's Realtime API.

Node 22+ has WebSocket built in. On Node 18–21, install the `ws` package: `npm install ws`.

## Connect

```ts
import { MeshAPI, RealtimeError } from "meshapi-node-sdk";

const client = new MeshAPI({ baseUrl: "https://api.meshapi.ai", token: "rsk_..." });

const session = await client.realtime.connect({ model: "openai/gpt-realtime-mini" });
```

## Configure the session

```ts
await session.send({
  type: "session.update",
  session: {
    type:         "realtime",
    modalities:   ["audio", "text"],
    voice:        "alloy",
    instructions: "You are a helpful assistant.",
  },
});
```

## Send audio

```ts
// pcmBytes is a Uint8Array of raw 16-bit PCM at 24 kHz mono
await session.sendAudio(pcmBytes);
```

## Async iterator

The async iterator yields `RealtimeMessage` objects and throws `RealtimeError` on server error envelopes:

```ts
try {
  for await (const msg of session) {
    if (msg.audio) {
      playAudio(msg.audio);           // Uint8Array
    } else if (msg.event) {
      const type = msg.event.type as string;
      if (type === "response.text.delta") {
        process.stdout.write(msg.event.delta as string);
      } else if (type === "response.done") {
        break;
      }
    }
  }
} finally {
  await session.close();
}
```

## Callback style

Use `on()` when you prefer event handlers over iteration:

```ts
session
  .on("message", (msg) => {
    if (msg.audio) playAudio(msg.audio);
    else if (msg.event?.type === "response.text.delta") {
      process.stdout.write(msg.event.delta as string);
    }
  })
  .on("error", (err) => console.error("error:", err))
  .on("close", (code, reason) => console.log("closed", code, reason));
```

## Close

`close()` returns a `Promise` that resolves when the TCP close handshake completes:

```ts
await session.close();
```

## Error handling

```ts
try {
  for await (const msg of session) { ... }
} catch (err) {
  if (err instanceof RealtimeError) {
    console.error("code:", err.code);     // "invalid_api_key", "insufficient_quota", …
    console.error("message:", err.message);
  }
}
```

## Full voice agent example

```ts
const session = await client.realtime.connect({ model: "openai/gpt-realtime-mini" });

await session.send({
  type: "session.update",
  session: { type: "realtime", modalities: ["audio"], voice: "alloy" },
});

// Stream microphone audio
(async () => {
  for await (const chunk of micStream) {
    await session.sendAudio(chunk);
  }
  await session.send({ type: "input_audio_buffer.commit" });
  await session.send({ type: "response.create" });
})();

// Play response audio
for await (const msg of session) {
  if (msg.audio) speaker.write(msg.audio);
}
await session.close();
```

## Supported models

| Model ID                   |
| -------------------------- |
| `openai/gpt-realtime-mini` |
| `openai/gpt-realtime`      |
| `openai/gpt-realtime-2`    |