Tool calling

Research prototypes came first: ReAct (Yao et al., 2022) interleaved reasoning traces with actions, and Toolformer (Schick et al., 2023) taught a model in a self-supervised way where to insert API calls. Commercial APIs standardised the pattern when OpenAI shipped function calling in June 2023, followed by Anthropic, Google and open-weight model families. Under the hood the tool list, typically names, descriptions and JSON Schema parameter definitions, is rendered into the prompt through the model's chat template, and fine-tuning teaches the model to emit a call in a reserved format instead of prose when a tool would help.

The wire protocol is a two-step exchange. In Anthropic's Messages API the assistant turn contains a tool_use block with an id, the tool name and an input object, and the response's stop_reason is tool_use; the application executes the call and sends a user turn with a tool_result block referencing that id, optionally flagged is_error. OpenAI's equivalent returns tool_calls whose arguments arrive as a JSON-encoded string, answered by a message with role tool and a matching tool_call_id. Both support several calls in one turn (parallel tool calls) and a tool_choice setting that lets the developer allow, force or forbid tool use. The loop continues until the model answers without calling a tool, which is exactly the control loop of an AI agent.

Reliability depends on the schema and the decoder. Without constraints a model can produce malformed JSON, invent parameters or pick the wrong tool; OpenAI's strict mode (August 2024) and similar grammar-constrained decoding guarantee schema-conformant arguments, though not correct ones. Every tool definition costs context tokens on every request, and accuracy in choosing tools degrades as the catalogue grows into the dozens, which is why larger systems load tools on demand or route first. Errors should be returned to the model as results rather than thrown, so it can repair its call; tool-use accuracy is benchmarked by suites such as the Berkeley Function Calling Leaderboard.

The security model is easy to get wrong. The model never executes anything; the application does, so the application is the enforcement point. Arguments are untrusted input to be validated and authorised against the end user's permissions rather than the service account's, and side-effecting tools need idempotency and, for high-impact actions, human confirmation. Tool results are untrusted too: text returned from a web fetch or an email tool is the main vector for indirect prompt injection. OWASP's Top 10 for LLM Applications 2025 covers the resulting risks as LLM06 Excessive Agency and LLM05 Improper Output Handling. Structured output is the non-executing sibling; the Model Context Protocol standardises how tools are described and served across applications.