Evaluate

POST /v1/evaluate answers structured questions about a piece of application state. Instead of prose you have to parse, each answer comes back typed — a chosen option with a probability for every option, a score against a scale you define, or a single probability.

POST https://api.meshapi.ai/v1/evaluate
Authorization: Bearer rsk_<your_key>

Use it for the decisions an application makes over and over: which queue a ticket belongs in, how urgent a message is, whether something needs a human. One request can ask several questions about the same state.

An ordinary API key calls this endpoint. If you have restricted a key to a specific set of scopes, that set has to include evaluate:write — a key with no scope restrictions already has it.

How it differs from structured output

Structured output makes a chat model return JSON in a shape you specify. It is the right tool when you want the model to produce content in a fixed format. Evaluate is for decisions. You give it the options, and it returns how likely each one is — a calibrated number you can threshold, route on, or log. You never write a prompt asking for JSON, and there is nothing to parse or repair.

	Structured output	Evaluate
You supply	A JSON schema	The questions and their options
You get back	JSON as a string in `message.content`	Typed answers, already parsed
Confidence	Not available	A probability per option
Failure mode	Model ignores the schema, returns prose	Not possible — the shape is fixed

Models

Model	Price	Context
`typesafe/jev-1.13`	$0.042 / 1M input tokens, output free	32,000 tokens

Question types

Every question declares a type, and that decides which criteria shape is legal.

`type`	`criteria`	Answer
`choice`	An object of `{your_key: description}`	The winning key, plus a probability for every key
`score`	An ordered array, lowest first	A number on that scale, with the scale echoed back as a legend
`noul`	Not allowed	A single probability between 0 and 1

Making a request

state is whatever your application looks like — a string, an object, or an array. Every question is answered against that same state.

curl
Python

curl https://api.meshapi.ai/v1/evaluate \
  -H "Authorization: Bearer rsk_<your_key>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "typesafe/jev-1.13",
    "state": {
      "subject": "Refund for order 4821 never arrived",
      "plan": "pro"
    },
    "questions": {
      "department": {
        "type": "choice",
        "instructions": "Which team should handle this ticket?",
        "criteria": {
          "billing": "Payments, refunds and invoices",
          "technical": "Bugs, outages and integration problems"
        }
      },
      "urgency": {
        "type": "score",
        "instructions": "How urgently does this need a reply?",
        "criteria": ["Can wait a week", "Within a day", "Immediately"]
      },
      "needs_human": {
        "type": "noul",
        "instructions": "Probability that this needs a human reply."
      }
    }
  }'

import httpx

resp = httpx.post(
    "https://api.meshapi.ai/v1/evaluate",
    headers={"Authorization": "Bearer rsk_<your_key>"},
    json={
        "model": "typesafe/jev-1.13",
        "state": {
            "subject": "Refund for order 4821 never arrived",
            "plan": "pro",
        },
        "questions": {
            "department": {
                "type": "choice",
                "instructions": "Which team should handle this ticket?",
                "criteria": {
                    "billing": "Payments, refunds and invoices",
                    "technical": "Bugs, outages and integration problems",
                },
            },
            "urgency": {
                "type": "score",
                "instructions": "How urgently does this need a reply?",
                "criteria": ["Can wait a week", "Within a day", "Immediately"],
            },
            "needs_human": {
                "type": "noul",
                "instructions": "Probability that this needs a human reply.",
            },
        },
    },
)

answers = resp.json()["answers"]
if answers["needs_human"]["noul"] > 0.7:
    escalate(to=answers["department"]["choice"])

You name the questions yourself, and those names come back as the keys of answers.

The response

{
  "id": "eval_2f9c4a1b8d3e5f7a6c0b9d2e",
  "model": "typesafe/jev-1.13",
  "answers": {
    "department": {
      "type": "choice",
      "choice": "billing",
      "probabilities": { "billing": 1, "technical": 0 },
      "confidence": 1
    },
    "urgency": {
      "type": "score",
      "score": 1.3,
      "legend": { "0": "Can wait a week", "1": "Within a day", "2": "Immediately" },
      "probabilities": { "0": 0.01, "1": 0.68, "2": 0.31 },
      "confidence": 0.53
    },
    "needs_human": { "type": "noul", "noul": 0.8 }
  },
  "usage": { "input_tokens": 396, "output_tokens": 64, "total_tokens": 460 }
}

noul is a probability, not a true/false. It is a number between 0 and 1, and 0.8 means “probably” — not “yes”. Compare it against a threshold you choose; treating it as a boolean makes every non-zero answer true.

A score legend is keyed by position in the array you sent. "1" means the second entry of criteria. Reordering that array therefore changes what a score means, silently and for every past comparison — treat the order as part of your schema.

Request rules

Anything the endpoint cannot make sense of is rejected with a 422 before the model is called, so you are never billed for a question that could not be answered.

Field	Rules
`model`	Required. An evaluation model id — see Models.
`state`	Required. A string, object, or array.
`questions`	Required, at least one. Each key is your own name for the question: letters, digits, `_` and `-`, up to 64 characters.
`instructions`	Required on every question, and cannot be blank.
`criteria`	At least two entries on `choice` and `score`. No blank keys, descriptions, or scale points. Not allowed on `noul`.

Two options is the floor because a single-option choice is not a decision — the model can only pick the one you gave it, and you would pay input tokens for an answer that was never in doubt. Blank criteria are refused for the same reason: a scale with no labels produces a number with no meaning. Fields this endpoint does not define are rejected rather than ignored, so a misspelled criteria fails loudly instead of quietly changing the question.

Attribution and limits

Two optional fields let you attribute the request in your own usage reports. Neither is ever forwarded to the model provider.

Field	What it does
`tags`	Your own labels, as an object of up to 16 string pairs. Filter and group spend by them in usage reports — useful when several apps share one key.
`safety_identifier`	Your own id for the end user the request was made for, so spend can be attributed per user.

Rate limits, spend caps, and credit balance apply exactly as they do on /v1/chat/completions. There is no streaming on this endpoint: an evaluation returns one small JSON body, so there is nothing to stream.

Errors

Status	Meaning
`400`	The model does not exist or cannot serve evaluations, or your request is larger than the model’s context window
`401` / `403`	Missing or invalid key; suspended key, or one without `evaluate:write`
`402`	Not enough credits
`422`	The request body failed validation — see Request rules
`429`	Rate limit reached
`503` / `504`	The provider returned an error or did not respond in time

Introduction

Accounts & Access

Core Inference

Retrieval & Search

Multimodal

Operations

Mesh CLI

Platform

Reference

How it differs from structured output

Models

Question types

Making a request

The response

Request rules

Attribution and limits

Errors

​How it differs from structured output

​Models

​Question types

​Making a request

​The response

​Request rules

​Attribution and limits

​Errors

How it differs from structured output

Models

Question types

Making a request

The response

Request rules

Attribution and limits

Errors