Skip to main content
Auto Routing (model: "auto") classifies each request and picks a model for you. Most confusion is about which model ran, what you were billed, and where it’s supported. See Auto Routing for the full guide.

​
‘auto’ didn’t route on my endpoint

Auto Routing only works on a few inference endpoints: Setting model: "auto" on any other endpoint won’t route. Use a concrete model ID there. POST /v1/embeddings rejects "auto" with 400 unprocessable_entity.

​
Which model actually served my request?

Mesh injects the resolved model into the response, but where depends on streaming:
  • Non-streaming: in the body, as x_resolved_model_id and x_auto_routed.
  • Streaming (stream: true): as HTTP response headers sent before the SSE stream, X-Auto-Routed and X-Resolved-Model-Id. They are not in the body, which is why people miss them on streams.
POST /v1/messages is the exception. A non-streaming Messages response echoes "model": "auto" and drops the x_auto_routed and x_classifier_usage fields, so only a streaming request reports the pick (in X-Resolved-Model-Id). For a non-streaming one, check Dashboard → Logs.

​
It keeps picking a cheap/default model (e.g. qwen3.5-27b)

The router never blocks a request on its own failure. Its algorithms run as a waterfall; if every one of them declines, times out or errors, it falls back to a reliable default, currently qwen/qwen3.5-27b. Note this is not a classifier model. The classifiers (for example openai/gpt-4o-mini) decide; the default is what serves the request when every algorithm missed. Landing on one is a very different signal from landing on the other. Check x_classifier_usage for the classifiers that ran. Check the response for:
If you always land on the fallback, routing is failing, not your prompt. A reason of entitlement_filtered, exclude_filtered or capability_filtered means the router’s pick wasn’t usable for this request (your model policy, your exclude_models, or tools) and it chose another model. See Fallback behavior. Short, conversational prompts (greetings, one-line questions) can also go to a dedicated fast-lane model without any fallback. That’s the heuristic algorithm, not a failure.

​
Costs are higher than the resolved model alone

With Auto Routing you’re billed for two things: the tokens of the resolved model plus the tokens consumed by the internal classifier model. Both show up in the usage dashboard. For cost-sensitive or high-volume paths, pin a specific model instead.

​
I need control over the model choice

auto gives the gateway full discretion. If you need determinism:
  • Pin a single model ID, or
  • Supply a models fallback array to constrain the set of models a request may use, instead of "auto".

​
First-token latency went up

The classification step adds an extra internal LLM call before your request is forwarded, which increases time-to-first-token. Two cases skip it: trivial conversational prompts answered by the fast lane, and a prompt classified in the last 24 hours, whose classification is cached. For latency-sensitive interactive use, pin a model rather than using auto.

​
model: "auto" returned an error

​
Still stuck?

See the Mesh API error reference or email contact@meshapi.ai.