> ## Documentation Index > Fetch the complete documentation index at: https://developers.meshapi.ai/llms.txt > Use this file to discover all available pages before exploring further. # Evaluate > Ask a model a typed question and get a calibrated probability back instead of prose to parse — choices, scores, and likelihoods from POST /v1/evaluate. `POST /v1/evaluate` answers structured questions about a piece of application state. Instead of prose you have to parse, each answer comes back typed — a chosen option with a probability for every option, a score against a scale you define, or a single probability. ```http theme={null} POST https://api.meshapi.ai/v1/evaluate Authorization: Bearer rsk_ ``` Use it for the decisions an application makes over and over: which queue a ticket belongs in, how urgent a message is, whether something needs a human. One request can ask several questions about the same state. An ordinary [API key](/docs/getting-started/api-keys) calls this endpoint. If you have restricted a key to a specific set of scopes, that set has to include **`evaluate:write`** — a key with no scope restrictions already has it. ## How it differs from structured output [Structured output](/docs/capabilities/structured-output) makes a chat model return JSON in a shape you specify. It is the right tool when you want the model to *produce content* in a fixed format. Evaluate is for *decisions*. You give it the options, and it returns how likely each one is — a calibrated number you can threshold, route on, or log. You never write a prompt asking for JSON, and there is nothing to parse or repair. | | Structured output | Evaluate | | ------------ | --------------------------------------- | --------------------------------- | | You supply | A JSON schema | The questions and their options | | You get back | JSON as a string in `message.content` | Typed answers, already parsed | | Confidence | Not available | A probability per option | | Failure mode | Model ignores the schema, returns prose | Not possible — the shape is fixed | ## Models | Model | Price | Context | | ------------------- | ------------------------------------------ | ------------- | | `typesafe/jev-1.13` | **\$0.042 / 1M input tokens**, output free | 32,000 tokens | ## Question types Every question declares a `type`, and that decides which `criteria` shape is legal. | `type` | `criteria` | Answer | | -------- | -------------------------------------- | -------------------------------------------------------------- | | `choice` | An object of `{your_key: description}` | The winning key, plus a probability for every key | | `score` | An ordered array, lowest first | A number on that scale, with the scale echoed back as a legend | | `noul` | Not allowed | A single probability between 0 and 1 | ## Making a request `state` is whatever your application looks like — a string, an object, or an array. Every question is answered against that same state. ```bash theme={null} curl https://api.meshapi.ai/v1/evaluate \ -H "Authorization: Bearer rsk_" \ -H "Content-Type: application/json" \ -d '{ "model": "typesafe/jev-1.13", "state": { "subject": "Refund for order 4821 never arrived", "plan": "pro" }, "questions": { "department": { "type": "choice", "instructions": "Which team should handle this ticket?", "criteria": { "billing": "Payments, refunds and invoices", "technical": "Bugs, outages and integration problems" } }, "urgency": { "type": "score", "instructions": "How urgently does this need a reply?", "criteria": ["Can wait a week", "Within a day", "Immediately"] }, "needs_human": { "type": "noul", "instructions": "Probability that this needs a human reply." } } }' ``` ```python theme={null} import httpx resp = httpx.post( "https://api.meshapi.ai/v1/evaluate", headers={"Authorization": "Bearer rsk_"}, json={ "model": "typesafe/jev-1.13", "state": { "subject": "Refund for order 4821 never arrived", "plan": "pro", }, "questions": { "department": { "type": "choice", "instructions": "Which team should handle this ticket?", "criteria": { "billing": "Payments, refunds and invoices", "technical": "Bugs, outages and integration problems", }, }, "urgency": { "type": "score", "instructions": "How urgently does this need a reply?", "criteria": ["Can wait a week", "Within a day", "Immediately"], }, "needs_human": { "type": "noul", "instructions": "Probability that this needs a human reply.", }, }, }, ) answers = resp.json()["answers"] if answers["needs_human"]["noul"] > 0.7: escalate(to=answers["department"]["choice"]) ``` You name the questions yourself, and those names come back as the keys of `answers`. ## The response ```json theme={null} { "id": "eval_2f9c4a1b8d3e5f7a6c0b9d2e", "model": "typesafe/jev-1.13", "answers": { "department": { "type": "choice", "choice": "billing", "probabilities": { "billing": 1, "technical": 0 }, "confidence": 1 }, "urgency": { "type": "score", "score": 1.3, "legend": { "0": "Can wait a week", "1": "Within a day", "2": "Immediately" }, "probabilities": { "0": 0.01, "1": 0.68, "2": 0.31 }, "confidence": 0.53 }, "needs_human": { "type": "noul", "noul": 0.8 } }, "usage": { "input_tokens": 396, "output_tokens": 64, "total_tokens": 460 } } ``` **`noul` is a probability, not a true/false.** It is a number between 0 and 1, and `0.8` means "probably" — not "yes". Compare it against a threshold you choose; treating it as a boolean makes every non-zero answer true. **A `score` legend is keyed by position in the array you sent.** `"1"` means the second entry of `criteria`. Reordering that array therefore changes what a score means, silently and for every past comparison — treat the order as part of your schema. ## Request rules Anything the endpoint cannot make sense of is rejected with a `422` before the model is called, so you are never billed for a question that could not be answered. | Field | Rules | | -------------- | ---------------------------------------------------------------------------------------------------------------------- | | `model` | Required. An evaluation model id — see [Models](#models). | | `state` | Required. A string, object, or array. | | `questions` | Required, at least one. Each key is your own name for the question: letters, digits, `_` and `-`, up to 64 characters. | | `instructions` | Required on every question, and cannot be blank. | | `criteria` | At least **two** entries on `choice` and `score`. No blank keys, descriptions, or scale points. Not allowed on `noul`. | Two options is the floor because a single-option choice is not a decision — the model can only pick the one you gave it, and you would pay input tokens for an answer that was never in doubt. Blank criteria are refused for the same reason: a scale with no labels produces a number with no meaning. Fields this endpoint does not define are **rejected rather than ignored**, so a misspelled `criteria` fails loudly instead of quietly changing the question. ## Attribution and limits Two optional fields let you attribute the request in your own usage reports. Neither is ever forwarded to the model provider. | Field | What it does | | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | | `tags` | Your own labels, as an object of up to 16 string pairs. Filter and group spend by them in usage reports — useful when several apps share one key. | | `safety_identifier` | Your own id for the end user the request was made for, so spend can be attributed per user. | [Rate limits](/docs/getting-started/rate-limits), spend caps, and credit balance apply exactly as they do on `/v1/chat/completions`. There is **no streaming** on this endpoint: an evaluation returns one small JSON body, so there is nothing to stream. ## Errors | Status | Meaning | | ------------- | --------------------------------------------------------------------------------------------------------------- | | `400` | The model does not exist or cannot serve evaluations, or your request is larger than the model's context window | | `401` / `403` | Missing or invalid key; suspended key, or one without `evaluate:write` | | `402` | Not enough credits | | `422` | The request body failed validation — see [Request rules](#request-rules) | | `429` | Rate limit reached | | `503` / `504` | The provider returned an error or did not respond in time |