Local Function Calling

Let a local model request function calls with a tools schema, then run them and return the results.

TL;DR

  1. Send a tools schema array in the chat request.
  2. The model returns tool_calls, not plain text output.
  3. Your app runs the function and feeds the result to /api/chat.

Define Tools

    tools array

    Describe callable functions as a tools array.

    "tools": [ { "type": "function", ... } ]
    Function Schema

    Give each tool a name and description.

    { "name": "get_weather",
      "description": "Get weather" }
    Parameter Types

    Define argument names and JSON types.

    "properties": {"city": {"type":"string"}}
    Required Fields

    List required parameter names in a required array.

    "required": ["city"]

Make The Call

    Pass tools

    Send the tools array to /api/chat.

    # body: { model, messages, tools }
    tool_calls

    The reply contains tool calls, not text.

    # message.tool_calls[0].function.name
    Arguments

    Each call carries parsed argument values.

    # function.arguments -> {"city":"Oslo"}
    Multiple Tools

    List several tools; the model picks which to call.

    "tools": [tool1, tool2]

Run And Return

    role tool

    Send the result back as a tool message.

    { "role": "tool", "content": "12C" }
    Final Answer

    The model turns the result into a reply.

    # Second call yields natural-language text
    Multiple Calls

    A reply can request more than one tool call at once.

    for call in message.tool_calls: ...

Python With ollama

    Define A Function

    Pass Python functions directly as tools.

    def get_weather(city: str) -> str: ...
    Pass tools

    The client builds the schema from functions.

    ollama.chat(model="llama3.1",
      messages=msgs, tools=[get_weather])
    Read tool_calls

    Inspect which tool the model chose.

    resp.message.tool_calls
    Multiple Functions

    Pass a list of functions; the client schemas each one.

    tools=[get_weather, get_time]

Tips

  1. Use a tool-capable model like llama3.1 for function calling, since smaller or older models may ignore the tools array.
  2. Describe each tool and its parameters clearly in the schema, because the model picks tools from your descriptions and types.

Warnings

  1. The model only names a function and arguments; your code must run it, then send the output back as a tool message.
  2. Always validate tool_calls arguments before executing, since a model can hallucinate values or call a tool you did not expect.

In Practice

FAQ