Guides

Tool Calling

View as Markdown

Tool calling lets a model ask your application to run a function and then continue with the result. MeshAPI implements the OpenAI-compatible tools / tool_choice interface on /v1/chat/completions, so existing OpenAI tool-calling code works unchanged.

The cycle

  1. You send a request with a tools array describing the functions available.
  2. The model replies with finish_reason: "tool_calls" and one or more tool_calls.
  3. You execute the function yourself and append the result as a role: "tool" message.
  4. You send the conversation back; the model produces its final answer.

1. Declare the tools

$curl https://api.meshapi.ai/v1/chat/completions \
>  -H "Authorization: Bearer <YOUR_RSK_KEY>" \
>  -H "Content-Type: application/json" \
>  -d '{
>    "model": "openai/gpt-4o-mini",
>    "messages": [{ "role": "user", "content": "What is the weather in Chennai?" }],
>    "tools": [
>      {
>        "type": "function",
>        "function": {
>          "name": "get_weather",
>          "description": "Get the current weather for a city.",
>          "parameters": {
>            "type": "object",
>            "properties": {
>              "city": { "type": "string", "description": "City name" }
>            },
>            "required": ["city"]
>          }
>        }
>      }
>    ]
>  }'

2. The model asks for a call

1{
2  "choices": [{
3    "finish_reason": "tool_calls",
4    "message": {
5      "role": "assistant",
6      "tool_calls": [{
7        "id": "call_abc123",
8        "type": "function",
9        "function": { "name": "get_weather", "arguments": "{\"city\":\"Chennai\"}" }
10      }]
11    }
12  }]
13}

arguments is a JSON-encoded string, not an object — parse it before use.

3. Return the result

Append the assistant message and a tool message carrying the matching tool_call_id:

1{
2  "model": "openai/gpt-4o-mini",
3  "messages": [
4    { "role": "user", "content": "What is the weather in Chennai?" },
5    {
6      "role": "assistant",
7      "tool_calls": [{
8        "id": "call_abc123",
9        "type": "function",
10        "function": { "name": "get_weather", "arguments": "{\"city\":\"Chennai\"}" }
11      }]
12    },
13    { "role": "tool", "tool_call_id": "call_abc123", "content": "{\"temp_c\": 31, \"condition\": \"humid\"}" }
14  ],
15  "tools": [ ... ]
16}

Keep sending tools on the follow-up. Every tool_call_id must be answered by exactly one tool message.

Controlling when tools are used

tool_choiceBehaviour
"auto" (default)Model decides
"none"Never call a tool
{"type": "function", "function": {"name": "..."}}Force a specific function

Tool calling disables the response cache

Any request carrying a tools array is never served from, or written to, the gateway response cache — a tool call depends on live state, so replaying a stored one would be wrong.

This is silent. If you are relying on caching for cost and you add tools to a request, that saving disappears with no error and no header. It is the correct behaviour, but worth knowing before it shows up as a bill.

Gemini thinking models

Gemini’s thinking models attach a thought signature to each tool call, and Vertex rejects a follow-up whose tool history is missing it. The OpenAI wire format has no slot for this, so MeshAPI exposes it in two places:

  • on the tool call itself, as tool_calls[].thought_signature
  • in a side channel, message.reasoning_details, for clients that strip unknown fields from tool_calls

Echo it back verbatim in the assistant message when you continue the conversation.

Some frameworks — LangChain among them — strip both fields automatically. MeshAPI detects the resulting rejection and retries once with the unsigned tool exchanges demoted to plain text, so your request still succeeds. You lose the thinking context on that turn, but you do not get an error. If you can preserve the signature, do; if your framework won’t, the fallback covers you.

  • Caching — why tools bypass the response cache
  • Structured Output — for shaping a response rather than calling code
  • The SDKs tab has per-language tool-calling examples for Python, Node.js, and Go