Auto Routing

Mesh API’s Auto Router allows you to simply set model: "auto" on any inference request. The gateway classifies the request, selects the most appropriate model from the live registry, and forwards the request transparently. This requires no client-side logic beyond setting the model field to "auto". For how the router makes the pick (the routing algorithms, the quality/cost/latency scoring formula, and the weight_profile knob that shifts it), see Routing Algorithms.

Supported endpoints

Auto Routing is supported across the following inference endpoints:

Endpoint	Streaming Supported
`POST /v1/chat/completions`	Yes
`POST /v1/responses`	Yes
`POST /v1/messages`	Yes

On POST /v1/messages, only streaming responses report the pick, in the X-Resolved-Model-Id header. A non-streaming Messages response echoes "model": "auto" and carries none of the x_auto_routed or x_classifier_usage fields. Check Dashboard → Logs to see which model served it. POST /v1/embeddings does not accept "auto". It returns 400 unprocessable_entity; name a specific embedding model.

Basic request

Just replace your specific model ID with "auto":

curl
Node.js SDK
Python SDK
Go SDK
Java SDK

curl https://api.meshapi.ai/v1/chat/completions \
  -H "Authorization: Bearer <YOUR_RSK_KEY>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "messages": [{"role": "user", "content": "Write a Python quicksort implementation"}]
  }'

const response = await client.chat.completions.create({
  model: "auto",
  messages: [{ role: "user", content: "Write a Python quicksort implementation" }],
});

from meshapi import ChatMessage, ChatCompletionParams, MeshAPI

client = MeshAPI(base_url="https://api.meshapi.ai", token="rsk_...")

response = client.chat.completions.create(
    ChatCompletionParams(
        model="auto",
        messages=[ChatMessage(role="user", content="Write a Python quicksort implementation")],
    )
)

model := "auto"
params := meshapi.ChatCompletionParams{
    Model: &model,
    Messages: []meshapi.ChatMessage{
        {Role: "user", Content: "Write a Python quicksort implementation"},
    },
}

_, err := client.Chat.Completions.Create(ctx, params)

ChatCompletionRequest request = ChatCompletionRequest.builder()
    .model("auto")
    .addMessage(ChatMessage.user("Write a Python quicksort implementation"))
    .build();

client.chat().completions().create(request);

Steering the choice

By default the router balances quality, cost and latency. Send weight_profile to lean it one way for a single request:

curl https://api.meshapi.ai/v1/chat/completions \
  -H "Authorization: Bearer rsk_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "weight_profile": "cost_first",
    "messages": [{"role": "user", "content": "Summarise this changelog in one line"}]
  }'

quality_first, balanced (default), cost_first and latency_first are the four profiles. You can also set a default on the API key, or on your team or organization, and still override it per request — see Routing Algorithms for the weights behind each profile and the full precedence order.

weight_profile applies to POST /v1/chat/completions. An unrecognised name falls back to balanced rather than erroring, so check the spelling if a profile doesn’t seem to be taking effect.

Response metadata

When a request is automatically routed, Mesh API injects metadata into the response so you know which model the router picked.

Non-streaming requests

The metadata is included directly in the response body.

{
  "id": "chatcmpl-...",
  "model": "anthropic/claude-haiku-4.5",
  "choices": [...],
  "x_auto_routed": true,
  "x_resolved_model_id": "anthropic/claude-haiku-4.5"
}

If the router could not make its own pick and fell back (see Fallback behavior), additional fields are present:

{
  "x_auto_routed": true,
  "x_resolved_model_id": "qwen/qwen3.5-27b",
  "x_auto_routed_fallback": true,
  "x_auto_routed_fallback_reason": "all_strategies_failed"
}

x_resolved_model_id is the router’s pick. If that model fails and the request moves to one of the router’s runner-ups, the model that served is in the body’s model field. Non-streaming responses also carry it in the X-Mesh-Routing-Model header, but only when the key’s routing policy enables retry or fallback.

Streaming requests

For streaming requests (stream: true), the metadata is included as HTTP response headers before the SSE stream begins:

X-Auto-Routed: true
X-Resolved-Model-Id: anthropic/claude-haiku-4.5

If a fallback was used, two further headers are present, mirroring the JSON fields above:

X-Auto-Routed-Fallback: true
X-Auto-Routed-Fallback-Reason: all_strategies_failed

X-Auto-Routed, X-Resolved-Model-Id and the fallback headers are not in the gateway’s CORS Access-Control-Expose-Headers list, so browser JavaScript calling the API cross-origin cannot read them. Server-side clients can.

Routing-data version

The data the router ranks models with is versioned. Every response to a model: "auto" request carries the version it routed on in the X-Mesh-Router-Version header. You can pin a version, v10 or v12 (v12 steers prompts that refer to an attached file or image to models that can read one). Highest precedence first:

The router_version body field
The X-Mesh-Router-Version request header
router_version in the API key’s routing policy
The key’s routing-policy template, then your organization’s default template

A value that isn’t a selectable version is skipped, not refused; with nothing selected, the gateway’s active version applies. A stored value on a key or template is validated when you save it.

Fallback behavior

The Auto Router is designed to never block a request due to its own failure. The routing algorithms run as a waterfall: each one that declines, times out or errors falls through to the next (see Routing Algorithms). If every algorithm misses, a configured default model serves the request. A pick made by any algorithm in the chain is a normal pick. The response carries x_auto_routed_fallback: true and an x_auto_routed_fallback_reason only when the router could not use its own pick. Common reasons:

Reason	Meaning
`all_strategies_failed`	Every algorithm missed; the default model served
`exception`	The router hit an internal error; the default model served
`entitlement_filtered`	The pick isn’t permitted by your model policy; another permitted model was chosen
`exclude_filtered`	The pick was in the request’s `exclude_models`; another model was chosen
`capability_filtered`	The request carries tools and the pick can’t call them; a tool-capable model was chosen
`no_entitled_candidates`	No routable model was left to re-pick from

You may also see classifier_timeout, classifier_error, invalid_response or empty_registry. Each means the router couldn’t classify the request and the default model served.

The classifier model and the default model are different models with different prices. A request that falls all the way through is served by the default — not by the classifier — so check x_resolved_model_id rather than assuming.

Billing

When using the Auto Router, you are billed for the tokens consumed by the resolved model that actually served the request, as well as the tokens consumed by the internal classifier model. Both appear in your usage dashboard. The classifier’s cost is broken out on the response itself, so you can see it per request rather than only in aggregate. On /v1/chat/completions, prompt_tokens, completion_tokens and total_tokens already include the classifier’s tokens; on /v1/responses only classifier_tokens is added, to total_tokens.

{
  "usage": {
    "prompt_tokens": 1458,
    "completion_tokens": 30,
    "total_tokens": 1488,
    "classifier_prompt_tokens": 1446,
    "classifier_completion_tokens": 26,
    "classifier_tokens": 1472
  },
  "x_classifier_usage": [
    { "model_id": "openai/gpt-4o-mini", "prompt_tokens": 1446, "completion_tokens": 26 }
  ]
}

x_classifier_usage is a list because a request whose first classifying algorithm misses bills for every classifier call it made, each as its own entry. A request answered by the heuristic fast lane makes no classifier call. A classification is cached for 24 hours per prompt, so a repeated prompt’s entry shows 0 tokens and isn’t billed.

When to use auto routing

Rapid prototyping — skip the model selection decision early in development
Mixed-complexity workloads — let the gateway route simple queries to cheap models and hard ones to frontier models
A/B testing — observe which models get selected for your actual traffic

Introduction

Accounts & Access

Core Inference

Retrieval & Search

Multimodal

Operations

Mesh Code

Mesh CLI

Platform

Reference

Supported endpoints

Basic request

Steering the choice

Response metadata

Non-streaming requests

Streaming requests

Routing-data version

Fallback behavior

Billing

When to use auto routing

​Supported endpoints

​Basic request

​Steering the choice

​Response metadata

​Non-streaming requests

​Streaming requests

​Routing-data version

​Fallback behavior

​Billing

​When to use auto routing

Supported endpoints

Basic request

Steering the choice

Response metadata

Non-streaming requests

Streaming requests

Routing-data version

Fallback behavior

Billing

When to use auto routing