Function calling is a protocol: you describe available functions with JSON Schema, the model replies with a structured tool_call — a name plus JSON arguments — and your validated backend executes the real work (Microsoft Graph, REST APIs, PowerShell, database queries, ticketing systems) before returning the result. The model never runs anything; it only fills in the form. This topic covers schemas, validation, chained calls and least-privilege execution.
Function calling is a protocol, not magic: you describe available functions in JSON Schema (name, description, typed parameters), send that list with the prompt, and the model responds with a structured tool_call — a function name plus JSON arguments — instead of prose. Your code executes the real function (a Graph API request, a REST call, a SQL query) and returns the result to the model, which then writes the final answer. The model never runs anything; it only fills in the form. That is exactly why validation on your side is mandatory: arguments are suggestions from a language model, not guarantees.
Tool calling is the bridge between “AI that talks about IT” and “AI that works in IT”. With it, one assistant can query Microsoft Graph for a device’s compliance state, hit a REST endpoint for license status, run a read-only PowerShell diagnostic, look up an error code in a database, and file a ServiceNow ticket — the five integrations that cover most helpdesk workflows. Without it, you copy-paste between the chat window and the admin center; with it, the model orchestrates and your validated tools do the touching.
Round trip in six beats: user asks → the LLM sees tool schemas → the model selects a tool and emits arguments → your backend validates and executes (Graph / REST / PowerShell / SQL / ticketing) → the result string goes back into the conversation as a “tool” message → the LLM produces the final answer, or chains the next tool. Design tips that separate good implementations: keep descriptions crisp (they are prompt content — “Gets an Intune device by device name including compliance state” beats “device stuff”); use strict schemas with enums for anything dangerous; return concise, structured results; and never let one turn chain more tool calls than your budget allows.
User: “Is LAP-042 compliant?”
↓
LLM + tool schemas
↓ tool_call JSON
Tool Selection → get_device(name)
↓
API / Function (Graph call)
↓ result JSON
Result → back to LLM
↓
Final Answer: “Compliant ✓ (last sync 09:14)”A working helpdesk flow: user pastes “LAP-042 gets 0x87D1041C on App X”. The model calls get_error_rule(“0x87D1041C”) → your rule database returns the detection-failure flow; calls get_device(“LAP-042”) → Graph returns the device and app install status; then drafts a ServiceNow incident with cause and next steps. In Microsoft-land the same pattern powers Copilot-style integrations over Graph (mail, Teams, Intune), while PowerShell tools let the agent run read-only scripts like Get-ComputerInfo through a constrained runspace rather than a raw shell.
JSON Schema defines each function: name, description and typed parameters. This schema IS the model’s documentation — invest in it, because selection quality is a direct function of description quality.
{"type": "function", "function": { "name": "get_intune_device", "description": "Get an Intune device by device name, including compliance state and last sync time", "parameters": { "type": "object", "properties": { "deviceName": { "type": "string", "description": "Exact device name, e.g. LAP-042" } }, "required": ["deviceName"] } } }
Send the schemas in the chat/completions call via the “tools” array (tool_choice: “auto”). Keep 3-10 tools per assistant — more than that and selection accuracy falls as schemas crowd the prompt.
The model returns tool_calls[0].function.name and .arguments (a JSON string). Parse, validate types/lengths/enums, check permissions, then execute the real call — a Graph request, REST endpoint, SQL query, or a constrained PowerShell runspace. Never execute unvalidated arguments.
Append a message with role “tool” containing the JSON result and resend the conversation. The model now answers, or chains the next tool. Forgetting this message is the classic bug: the model never saw the result and invents an outcome.
Persist name, arguments, caller and result for each execution. When (not if) an argument turns out wrong, this log is the difference between a five-minute fix and an unexplainable incident.
{ "name": "get_intune_device", "arguments": "{\"deviceName\": \"LAP-042\"}" }Treat every tool as a privilege escalation surface. Run tools under a least-privilege identity: a Graph app registration scoped to Device.Read-only today, ReadWrite only when a workflow truly needs it; a read-only SQL user, never the owner; PowerShell inside a constrained runspace with allow-listed cmdlets, never an open shell. Validate all arguments server-side (type, enum, length), rate-limit tool executions per user, and log name + arguments + result for audit. Remember indirect prompt injection too: content the model reads can instruct it to call a tool — topic 19 covers the defenses in depth.
It only selects and fills: given tool schemas and the conversation, it emits a tool_call — a function name plus JSON arguments. Execution is entirely your code, and the result returns to the model as a “tool” message so it can compose the final answer. This separation is what makes the pattern auditable: nothing runs unless your validated backend runs it.
Three levers: precise, task-scoped schemas with concrete descriptions; few tools per assistant (3-10) so selection stays unambiguous; and strict parameter schemas with enums and descriptions that pin formats. Beyond the schema, log and replay failures — most selection errors trace back to overlapping or vague tools.
Not as an open shell. Wrap specific scripts as named tools (“get_log_tail”, “get_service_status”) executed in a constrained runspace with an allow-listed cmdlet set, a read-only service identity, timeouts and output size caps, with every invocation logged. The model chooses which tool to call; it never writes script text that gets executed verbatim.
Wrap two real functions as tools — get_error_rule(code) against your topic-13 vector store, and get_intune_device(name) against Graph’s read-only API. Give the model both schemas and the question “why does LAP-042 fail App X with 0x87D1041C?” — and require it to use both tools before answering.
No — and the distinction is the whole security model. The model emits structured JSON describing a call; your backend validates and executes it with its own credentials and permissions. The model has no runtime, no shell and no network access of its own.
Yes on modern APIs: the model can request multiple tool calls in one turn — e.g. get_device and get_error_rule together. Execute independent calls concurrently, then return all results as separate “tool” messages. Watch ordering when calls depend on each other’s outputs.
Technically dozens; practically 3-10 for reliable selection. With large tool sets, group by scenario or add a routing step (“triage tools” vs “remediation tools”) — and expect accuracy to fall as schemas crowd the prompt.