Routing Algorithms

Auto Routing covers how to use model: "auto". This page covers how the pick is actually made, and the one knob you can turn.

Two layers

A routing decision goes through two layers, and the word “weight” means something different in each:

Which algorithm chain runs — traffic is split across named paths by traffic weights. Operator-configured; you observe the result, you don’t set it.
Which model that algorithm picks — the weighted algorithm scores every candidate with scoring weights over quality, cost, and latency. You can influence this per request, with weight_profile.

The algorithms

Each path is an ordered waterfall. Algorithms run in sequence and the first one that returns a model wins; an algorithm that declines (“abstains”) costs nothing and falls through to the next. Every algorithm has its own timeout, and none of them can fail your request — see Nothing here can fail a request.

Algorithm	How it picks	LLM call	Typical use
`heuristic`	String-gates trivial prompts (greetings, one-line questions) and routes straight to the general-conversation model	No	A fast lane in front of the others — no classify round-trip
`weighted`	Classifies, then scores the candidate pool on quality, cost and latency and takes the argmax	Yes (classifier)	The main selector
`benchmark`	Classifies into a task category, then takes the top-ranked model for that category	Yes (classifier)	Quality-led picking; also the `weighted` fallback
`registry`	Shows the full candidate list to a classifier, which picks by reading model descriptions	Yes (classifier)	Last resort when the category ranking has nothing

heuristic is deliberately conservative. Any hint of task work, recency, code, links, digits, or length over ~140 characters makes it decline, so the request takes the full path instead. Over-declining is harmless; a wrong fast-lane pick is not.

What the pool is

Every algorithm picks from the same candidate pool: the models the gateway currently serves for that surface, already filtered by your key’s model policy and the request’s capability requirements. An algorithm can only ever return a model you were entitled to call directly.

Traffic weights

The active configuration names one or more paths, each with a relative weight, and a path is chosen per request by weighted random.

[
  { "name": "weighted-first",  "weight": 90,
    "algorithms": [ { "name": "heuristic", "timeout_ms": 200 },
                    { "name": "weighted",  "timeout_ms": 5000 },
                    { "name": "benchmark", "timeout_ms": 5000 },
                    { "name": "registry",  "timeout_ms": 5000 } ] },
  { "name": "benchmark-first", "weight": 10,
    "algorithms": [ { "name": "heuristic", "timeout_ms": 200 },
                    { "name": "benchmark", "timeout_ms": 5000 },
                    { "name": "registry",  "timeout_ms": 5000 } ] }
]

Weights are relative positive integers, normalised internally — [90, 10] and [9, 1] are identical. The shape above is illustrative; the live split is an operating decision and changes without an API change.

Scoring weights

When the weighted algorithm runs, it scores every candidate in the pool:

score(m) = w_q · Q(m)  +  w_c · C(m)  +  w_l · L(m)

Highest score wins. The runners-up become the fallover chain, so if the winner fails the request moves down the same ranking rather than to an unrelated model.

The three signals

All three are normalised so higher is always better, which is why cost and latency are inverted — a cheap model scores high on C, a fast one scores high on L.

Signal	Source	Normalisation
Q — quality	The model’s position in the ranking for the classified task category. Candidates with a measured benchmark index (coding / math / general, chosen by category) are reordered within the slots their cohort already holds — best index first, ties broken on curated rank	`1 - pos/(N-1)` over the pool
C — cost	Blended list price per 1M tokens: `0.3 × prompt + 0.7 × completion`	Log-scaled, then min-max inverted
L — latency	Observed p95 for that model — time-to-first-token on streaming requests, full wall-clock otherwise	Min-max inverted

Only the order of a measured index matters, never its magnitude, and a candidate with no measured score keeps its curated position rather than dropping. Nothing falls out of the pool for want of a benchmark.

Missing signals degrade, they don’t error

A candidate missing a signal drops that term, and its remaining weights are renormalised to sum to 1. A candidate with neither cost nor latency data is scored on quality alone — so when the whole pool has no cost or latency data, weighted returns exactly what benchmark would have.

Weight profiles

The (w_q, w_c, w_l) vector is named. Four profiles ship:

Profile	w_q (quality)	w_c (cost)	w_l (latency)	Picks
`quality_first`	0.75	0.15	0.10	The strongest model that isn’t wildly expensive
`balanced` (default)	0.40	0.35	0.25	The knee of the quality/cost curve
`cost_first`	0.15	0.70	0.15	The cheapest model that clears a quality floor
`latency_first`	0.15	0.25	0.60	The fastest model that clears a quality floor

Because each candidate’s present weights are renormalised, only the ratio between w_q, w_c and w_l affects the outcome — not the absolute sum.

Choosing a profile

Send weight_profile on a chat completions request:

curl
Header
Python SDK

curl https://api.meshapi.ai/v1/chat/completions \
  -H "Authorization: Bearer rsk_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "weight_profile": "cost_first",
    "messages": [{"role": "user", "content": "Summarise this changelog in one line"}]
  }'

curl https://api.meshapi.ai/v1/chat/completions \
  -H "Authorization: Bearer rsk_YOUR_KEY" \
  -H "X-Mesh-Weight-Profile: latency_first" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "messages": [{"role": "user", "content": "Summarise this changelog in one line"}]
  }'

The header exists for proxies and gateways that cannot rewrite a JSON body. When both are sent, the body field wins — it is the parameter you wrote, the header is something an intermediary may have injected.

response = client.chat.completions.create(
    ChatCompletionParams(
        model="auto",
        weight_profile="cost_first",
        messages=[ChatMessage(role="user", content="Summarise this changelog in one line")],
    )
)

Precedence

The effective profile is resolved highest-first:

Request — the weight_profile body field, or the X-Mesh-Weight-Profile header
API key — the weight_profile entry in the key’s routing policy
Team / organization — the same entry on the key’s routing-policy template, else your org’s default template
Gateway default — balanced

So you can set a house default on the key and still override it for one request.

An unrecognised profile name is not an error. A typo like "cheapest" falls back to balanced and the request is served normally — a misspelt profile is silently the default rather than a 400. Check the spelling against the four names above if routing isn’t behaving the way you expect.

weight_profile applies to POST /v1/chat/completions only, and only while weighted is the algorithm serving the request. On /v1/responses and /v1/router/select it is ignored and the gateway default applies. Sending it is always safe — it is stripped before your request reaches the upstream provider.

Previewing a decision

POST /v1/router/select returns the model the Auto Router would pick, without running inference or billing you for one. Useful for pinning a model yourself, or for checking what a prompt classifies as.

curl https://api.meshapi.ai/v1/router/select \
  -H "Authorization: Bearer rsk_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{"role": "user", "content": "Write a Python quicksort implementation"}]
  }'

{
  "model": "anthropic/claude-haiku-4.5",
  "auto_router": { "fallback_used": false, "fallback_reason": null },
  "reasoning_effort": "medium"
}

Two optional fields narrow the pick: candidate_models restricts it to ids you name (intersected with your key’s policy — an empty intersection is a 422), and exclude_models removes ids from consideration, which is how an “ask another model” flow avoids re-picking the one already shown.

The classifier still runs, so a select call takes roughly as long as the routing stage of a real request. reasoning_effort is a hint and is null unless the benchmark algorithm classified the request.

Nothing here can fail a request

model: "auto" always resolves to a concrete model. Every layer is fail-soft by design:

An algorithm that declines, times out, or errors falls through to the next in the chain.
A chain that exhausts every algorithm falls through to the configured default model.
A missing scoring signal drops that term instead of dropping the candidate.
An unknown weight_profile — at any precedence tier — degrades to balanced.
A settings-store outage falls back to the built-in defaults rather than refusing to route.

Whenever a tier below the first is used, the response says so: x_auto_routed_fallback and x_auto_routed_fallback_reason on the body, or X-Auto-Routed-Fallback and X-Auto-Routed-Fallback-Reason on a stream. See Response metadata for the full shape, and Debug → Auto Routing when a pick surprises you.

Introduction

Accounts & Access

Core Inference

Retrieval & Search

Multimodal

Operations

Mesh CLI

Platform

Reference

Routing Algorithms

Two layers

The algorithms

What the pool is

Traffic weights

Scoring weights

The three signals

Missing signals degrade, they don’t error

Weight profiles

Choosing a profile

Precedence

Previewing a decision

Nothing here can fail a request

​Two layers

​The algorithms

​What the pool is

​Traffic weights

​Scoring weights

​The three signals

​Missing signals degrade, they don’t error

​Weight profiles

​Choosing a profile

​Precedence

​Previewing a decision

​Nothing here can fail a request

Two layers

The algorithms

What the pool is

Traffic weights

Scoring weights

The three signals

Missing signals degrade, they don’t error

Weight profiles

Choosing a profile

Precedence

Previewing a decision

Nothing here can fail a request