Batch API

The Batch API is for asynchronous, high-volume inference jobs where you do not need an answer immediately.

Only models whose catalog entry has supports_batching: true can be batched. Today these are OpenAI models (for example openai/gpt-5.4, openai/gpt-4o-mini). Check the flag in GET /v1/models. Any other model returns 400 model_not_supported_for_batching. Batches always run on Mesh’s platform credentials, never on your BYOK keys.

Workflow

Prepare a request bundle
Create a batch with POST /v1/batches
Poll GET /v1/batches/{batch_id} — results are included inline once complete

1 & 2. Create the batch

Pass your requests inline — no separate file upload required.

curl
Node.js SDK
Python SDK
Go SDK
Java SDK

curl https://api.meshapi.ai/v1/batches \
  -H "Authorization: Bearer <YOUR_RSK_KEY>" \
  -H "Content-Type: application/json" \
  -d '{
    "requests": [
      {
        "custom_id": "doc-001",
        "body": {
          "model": "openai/gpt-5.4",
          "messages": [{ "role": "user", "content": "Summarize this note." }]
        }
      }
    ],
    "completion_window": "24h"
  }'

const batch = await client.batches.create({
  requests: [
    {
      custom_id: "doc-001",
      body: {
        model: "openai/gpt-5.4",
        messages: [{ role: "user", content: "Summarize this note." }],
      },
    },
  ],
  completion_window: "24h",
});

from meshapi import BatchRequestItem, CreateBatchParams, MeshAPI

client = MeshAPI(base_url="https://api.meshapi.ai", token="rsk_...")

batch = client.batches.create(
    CreateBatchParams(
        requests=[
            BatchRequestItem(
                custom_id="doc-001",
                body={
                    "model": "openai/gpt-5.4",
                    "messages": [{"role": "user", "content": "Summarize this note."}],
                },
            )
        ],
        completion_window="24h",
    )
)

window := "24h"
batch, err := client.Batches.Create(ctx, meshapi.CreateBatchParams{
    Requests: []meshapi.BatchRequestItem{
        {
            CustomID: "doc-001",
            Body: map[string]interface{}{
                "model": "openai/gpt-5.4",
                "messages": []map[string]string{
                    {"role": "user", "content": "Summarize this note."},
                },
            },
        },
    },
    CompletionWindow: &window,
})

var item = new com.meshapi.sdk.types.batch.BatchRequestItem();
item.customId = "doc-001";
item.body = java.util.Map.of(
    "model", "openai/gpt-5.4",
    "messages", java.util.List.of(java.util.Map.of("role", "user", "content", "Summarize this note."))
);

var batchRequest = new com.meshapi.sdk.types.batch.CreateBatchRequest();
batchRequest.requests = java.util.List.of(item);
batchRequest.completionWindow = "24h";

var batch = client.batches().create(batchRequest);

Request item fields

Each entry in requests supports:

Field	Type	Notes
`custom_id`	string	Your identifier, echoed back on the matching result. Required.
`body`	object	The request body (e.g. a chat completion payload). Required.
`method`	string	HTTP method. Defaults to `POST`.
`url`	string	Defaults to `/v1/chat/completions`, the only supported value. Every batch runs against chat completions.

The batch create call also accepts an optional metadata object (arbitrary key-value pairs) alongside completion_window.

Limits

A batch may not mix models — all requests must target the same model, or the create call returns 400 mixed_models.
You can have at most 10 batches in a non-terminal state at once; an eleventh returns 429 batch_limit_exceeded.

3. Poll and read results

Poll until status is a terminal value. When completed, the response includes a results array — no separate file download needed.

curl https://api.meshapi.ai/v1/batches/batch_xyz789 \
  -H "Authorization: Bearer <YOUR_RSK_KEY>"

{
  "id": "batch_xyz789",
  "status": "completed",
  "request_counts": { "total": 1, "completed": 1, "failed": 0 },
  "results": [
    {
      "custom_id": "doc-001",
      "response": {
        "status_code": 200,
        "body": {
          "choices": [{ "message": { "content": "The note says..." } }],
          "usage": { "prompt_tokens": 12, "completion_tokens": 40 }
        }
      },
      "error": null
    }
  ]
}

Requests that failed are not in results; they come back in a separate errors_detail array, each entry carrying its custom_id and the error. Both arrays are only returned once the batch is completed. Common statuses:

validating
in_progress
finalizing
completed
failed
cancelling
cancelled
expired

Notes

All requests in a batch must use the same model.
Batch jobs are best for throughput, not low-latency interactive use.
Use GET /v1/batches to list recent batches and POST /v1/batches/{batch_id}/cancel to cancel one.
Results are matched by custom_id — the output order is not guaranteed.

Batch statuses

Status	Meaning
`validating`	Submitted — file is being validated before processing starts
`in_progress`	Processing — keep polling
`finalizing`	Processing complete — output file is being assembled
`completed`	All requests finished
`failed`	Batch-level failure. The reason is in the batch’s `errors` object; there is no `results` or `errors_detail`
`cancelling`	Cancellation requested — winding down
`cancelled`	Cancelled before completion
`expired`	Batch exceeded the completion window and was not finished

Cancelling a batch

curl -X POST https://api.meshapi.ai/v1/batches/batch_01abc.../cancel \
  -H "Authorization: Bearer rsk_YOUR_KEY"

Returns the updated batch object. The status transitions to cancelling and eventually cancelled. Partial results may still be available.

Listing batches

curl https://api.meshapi.ai/v1/batches \
  -H "Authorization: Bearer rsk_YOUR_KEY"

Supports cursor-based pagination via after (batch ID) and limit (1–100, default 20) query parameters. List rows are a summary: id, status, model, file IDs and timestamps. They don’t carry request_counts or results; fetch a batch by ID for those.

When to use batching

Large-scale data processing — summarizing, classifying, or translating thousands of documents
Overnight jobs — non-urgent workloads that can run during off-peak hours
Throughput without client-side orchestration — submit thousands of requests in one call. Batch requests are billed at the model’s standard (on-demand) rates, not a discounted batch rate

Introduction

Accounts & Access

Core Inference

Retrieval & Search

Multimodal

Operations

Mesh Code

Mesh CLI

Platform

Reference

Workflow

1 & 2. Create the batch

Request item fields

Limits

3. Poll and read results

Notes

Batch statuses

Cancelling a batch

Listing batches

When to use batching

​Workflow

​1 & 2. Create the batch

​Request item fields

​Limits

​3. Poll and read results

​Notes

​Batch statuses

​Cancelling a batch

​Listing batches

​When to use batching

Workflow

1 & 2. Create the batch

Request item fields

Limits

3. Poll and read results

Notes

Batch statuses

Cancelling a batch

Listing batches

When to use batching