Function Calling with Ollama: Building Local Tool-Using AI Agents

Ollama supports function calling (also called tool calling) for local models, letting a model decide to call a function, read the result, and keep working toward a goal instead of just returning text. This tutorial is almost entirely code: pull a model, define a tool, call it, and wire up a minimal agent loop.

Prerequisites

# install Ollama (macOS)
brew install ollama
ollama serve &

# or Linux
curl -fsSL https://ollama.com/install.sh | sh

Step 1: Pull a Tool-Capable Model

Not every model supports tool calling. Llama 3.1/3.2, Qwen2.5, Mistral Nemo, and Firefunction-v2 do.

ollama pull llama3.1
ollama pull qwen2.5

Step 2: Define a Tool Schema

A tool is a JSON Schema description of a function: name, description, and parameters.

{
  "type": "function",
  "function": {
    "name": "get_weather",
    "description": "Get the current weather for a city",
    "parameters": {
      "type": "object",
      "properties": {
        "city": { "type": "string", "description": "City name" },
        "unit": { "type": "string", "enum": ["celsius", "fahrenheit"] }
      },
      "required": ["city"]
    }
  }
}

Step 3: Call the Chat API with curl

curl http://localhost:11434/api/chat -d '{
  "model": "llama3.1",
  "messages": [
    { "role": "user", "content": "What is the weather in Boston?" }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get the current weather for a city",
        "parameters": {
          "type": "object",
          "properties": {
            "city": { "type": "string" },
            "unit": { "type": "string", "enum": ["celsius", "fahrenheit"] }
          },
          "required": ["city"]
        }
      }
    }
  ],
  "stream": false
}'

The response includes a tool_calls array instead of a plain content string when the model decides to call your function:

{
  "message": {
    "role": "assistant",
    "content": "",
    "tool_calls": [
      {
        "function": {
          "name": "get_weather",
          "arguments": { "city": "Boston", "unit": "fahrenheit" }
        }
      }
    ]
  }
}

Step 4: The Same Call in Python

import ollama

response = ollama.chat(
    model="llama3.1",
    messages=[{"role": "user", "content": "What is the weather in Boston?"}],
    tools=[
        {
            "type": "function",
            "function": {
                "name": "get_weather",
                "description": "Get the current weather for a city",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "city": {"type": "string"},
                        "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
                    },
                    "required": ["city"],
                },
            },
        }
    ],
)

print(response["message"].get("tool_calls"))

Step 5: Execute the Tool and Send the Result Back

Ollama does not execute functions for you. You read tool_calls, run your own Python function, then append the result as a tool message and call chat again.

def get_weather(city, unit="fahrenheit"):
    # replace with a real weather API call
    return {"city": city, "temp": 61, "unit": unit, "condition": "cloudy"}

available_functions = {"get_weather": get_weather}

messages = [{"role": "user", "content": "What is the weather in Boston?"}]

response = ollama.chat(model="llama3.1", messages=messages, tools=[...])
messages.append(response["message"])

for call in response["message"].get("tool_calls", []):
    fn = available_functions[call["function"]["name"]]
    result = fn(**call["function"]["arguments"])
    messages.append({
        "role": "tool",
        "content": str(result),
    })

final = ollama.chat(model="llama3.1", messages=messages)
print(final["message"]["content"])

Step 6: Full Minimal Agent Loop

Wrap steps 4 and 5 in a loop so the model can chain multiple tool calls before giving a final answer:

import ollama

def get_weather(city, unit="fahrenheit"):
    return {"city": city, "temp": 61, "unit": unit, "condition": "cloudy"}

TOOLS = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get the current weather for a city",
        "parameters": {
            "type": "object",
            "properties": {
                "city": {"type": "string"},
                "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
            },
            "required": ["city"],
        },
    },
}]

AVAILABLE = {"get_weather": get_weather}

def run_agent(prompt, model="llama3.1", max_turns=5):
    messages = [{"role": "user", "content": prompt}]
    for _ in range(max_turns):
        response = ollama.chat(model=model, messages=messages, tools=TOOLS)
        message = response["message"]
        messages.append(message)

        calls = message.get("tool_calls")
        if not calls:
            return message["content"]

        for call in calls:
            fn = AVAILABLE[call["function"]["name"]]
            result = fn(**call["function"]["arguments"])
            messages.append({"role": "tool", "content": str(result)})

    return "Max turns reached without a final answer."

print(run_agent("What is the weather in Boston?"))

Common Pitfalls

Small models skip the tool and answer from memory: use a model explicitly listed as tool-capable, and keep the tool description short and unambiguous.

Arguments arrive as a string instead of a dict on some models: call json.loads() on call[“function”][“arguments”] defensively before unpacking.

The loop never terminates: always cap max_turns, and return early the moment tool_calls is empty.

Ollama vs. Cloud-Managed Agents

Ollama’s tool calling gives you the raw building block: one request, one round of tool_calls, no built-in retries, memory, or sandboxing. That is a feature for local, cost-free experimentation, but for production-grade multi-step agents with hosted retries and event streaming, compare it against a managed offering before building a lot of custom orchestration around it.