How to Use Ollama Streaming Tool Calls for Real-Time AI APIs

Building AI-powered chat and API applications is evolving fast. With Ollama’s latest support for streaming responses and real-time tool (function/API) calls, developers can now create chatbots and API endpoints that feel truly interactive—responding instantly, calling external APIs, and delivering results as the conversation unfolds.

In this hands-on guide, you’ll learn:

• What streaming and tool calling mean in Ollama, and why they matter for API developers
• How to implement these features using cURL, Python, and JavaScript (Node.js)
• How Ollama handles tool parsing during streaming for robust, low-latency AI applications
• Tips for optimizing response quality and context
• How using Apidog can streamline your API testing and documentation workflow alongside Ollama

💡 Looking for a robust API platform that generates beautiful API Documentation and boosts your team’s productivity? Apidog offers a unified workspace for API design, testing, and collaboration—replacing Postman at a more affordable price.

What are Streaming Responses & Tool Calls in Ollama?

Streaming Responses: Faster, More Natural AI Conversations

Traditionally, chatbots waited to generate a full response before sending it to the user. With streaming, the model sends partial outputs as they’re generated—delivering text word-by-word or phrase-by-phrase, just like a real conversation.

This reduces wait times and improves user experience, especially for API-driven chat interfaces.

Tool Calling: Connecting AI to Real-World APIs

Tool calling enables your AI model to invoke custom functions or APIs on demand. You define tools (e.g., get_current_weather, calculate_sum), and the model decides when to call them based on user intent.

Example tools:

• get_current_weather(location): Fetch current weather for a city.
• search_web(query): Retrieve real-time web data.
• calculate_sum(a, b): Perform calculations.

You describe tool schemas to Ollama, and when the model sees an opportunity, it emits a tool call with arguments. Your backend executes the tool (API or function), returns the result, and the model continues generating its response, now informed by real data.

Why Combine Streaming & Tool Calling?

The real power comes from combining both features:

• Users see the AI “thinking” and responding in real time
• When a tool call is needed, your app intercepts it, executes the function/API, and seamlessly feeds results back into the conversation
• The model continues streaming its reply, now enhanced with up-to-date results

This enables highly responsive, capable AI assistants and developer-facing bots.

Prerequisites

To follow this tutorial, you’ll need:

• Ollama installed (latest version)
• Command-line experience (for cURL)
• Python 3.x with pip (for Python section)
• Node.js with npm (for JavaScript section)
• Familiarity with JSON data structures

Supported Ollama Models

Ollama currently supports streaming tool calls in several top models, including:

• Qwen 3
• Devstral
• Qwen2.5, Qwen2.5-coder
• Llama 3.1, Llama 4
• ...with more models added regularly

Step-by-Step: Streaming Tool Calls with cURL

Let’s walk through a simple tool call: asking for the weather in Toronto.

1. Define the Tool Schema

We’ll use a tool called get_current_weather:

• location (string): e.g., “Toronto”
• format (string): “celsius” or “fahrenheit”

2. Construct the cURL Request

curl http://localhost:11434/api/chat -d '{
  "model": "qwen3",
  "messages": [
    {
      "role": "user",
      "content": "What is the weather today in Toronto?"
    }
  ],
  "stream": true,
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_current_weather",
        "description": "Get the current weather for a location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "City name (e.g. San Francisco, CA)"
            },
            "format": {
              "type": "string",
              "description": "Temperature format: 'celsius' or 'fahrenheit'",
              "enum": ["celsius", "fahrenheit"]
            }
          },
          "required": ["location", "format"]
        }
      }
    }
  ]
}'

Key points:

• stream: true enables streaming output
• tools array defines available tools for the conversation

3. Run and Interpret the Output

After running the command, you’ll see JSON objects streamed to your terminal. Look for:

• "content": The AI’s partial response
• "tool_calls": Appears when a tool call is triggered

Sample tool call chunk:

{
  "message": {
    "role": "assistant",
    "content": "",
    "tool_calls": [
      {
        "function": {
          "name": "get_current_weather",
          "arguments": {
            "location": "Toronto",
            "format": "celsius"
          }
        }
      }
    ]
  },
  "done": false
}

When this appears:

• Pause the stream
• Execute your weather API with the arguments
• Send the result back to Ollama as a new tool message
• Ollama continues streaming its final response, integrating the result

Streaming Tool Calls with Python (Ollama SDK)

Ollama’s Python SDK makes tool integration seamless for backend engineers.

1. Install the SDK

pip install -U ollama

2. Define the Tool Function

import ollama

def add_two_numbers(a: int, b: int) -> int:
    """
    Add two numbers.
    """
    print(f"--- Tool 'add_two_numbers' called with a={a}, b={b} ---")
    return a + b

3. Streaming Chat with Tool Calls

messages = [{'role': 'user', 'content': 'What is three plus one?'}]

response_stream = ollama.chat(
    model='qwen3',
    messages=messages,
    tools=[{
        'type': 'function',
        'function': {
            'name': 'add_two_numbers',
            'description': 'Add two integer numbers together.',
            'parameters': {
                'type': 'object',
                'properties': {
                    'a': {'type': 'integer', 'description': 'The first number'},
                    'b': {'type': 'integer', 'description': 'The second number'}
                },
                'required': ['a', 'b']
            }
        }
    }],
    stream=True
)

for chunk in response_stream:
    if chunk['message']['content']:
        print(chunk['message']['content'], end='', flush=True)
    if 'tool_calls' in chunk['message'] and chunk['message']['tool_calls']:
        tool_call_info = chunk['message']['tool_calls'][0]
        print(f"\n--- Detected Tool Call: {tool_call_info['function']['name']} ---")
        break

4. Handling the Tool Call

When a tool call is detected:

• Extract tool name and arguments
• Call your Python function (e.g., add_two_numbers)
• Append a tool response message and continue the conversation

Tip: Always string-ify your result when sending it back.

Streaming Tool Calls with JavaScript (Node.js)

Integrate Ollama tool calls directly into your Node.js backend or API gateway.

1. Install the Ollama NPM Package

npm i ollama

2. Define Tool Schema and Execution Logic

import ollama from 'ollama';

const addTool = {
    type: 'function',
    function: {
        name: 'addTwoNumbers',
        description: 'Add two numbers together',
        parameters: {
            type: 'object',
            required: ['a', 'b'],
            properties: {
                a: { type: 'number', description: 'The first number' },
                b: { type: 'number', description: 'The second number' }
            }
        }
    }
};

function executeAddTwoNumbers(a, b) {
    console.log(`--- Tool 'addTwoNumbers' called with a=${a}, b=${b} ---`);
    return a + b;
}

3. Run the Streaming Conversation

const messages = [{ role: 'user', content: 'What is 2 plus 3?' }];

const responseStream = await ollama.chat({
    model: 'qwen3',
    messages: messages,
    tools: [addTool],
    stream: true
});

for await (const chunk of responseStream) {
    if (chunk.message.content) {
        process.stdout.write(chunk.message.content);
    }
    if (chunk.message.tool_calls && chunk.message.tool_calls.length > 0) {
        toolToCallInfo = chunk.message.tool_calls[0];
        process.stdout.write(`\n--- Detected Tool Call: ${toolToCallInfo.function.name} ---\n`);
        break;
    }
}

When a tool call is detected:

• Extract tool name and arguments
• Execute your JavaScript function (e.g., executeAddTwoNumbers)
• Send the result as a new tool message
• Continue streaming the AI’s response

How Ollama Streams and Parses Tool Calls

Ollama’s incremental parser recognizes tool call triggers as the model streams output. This means:

• You don’t have to wait for the entire response to parse for function calls
• Tool calls are identified and handled in real time, reducing latency and improving accuracy
• The system avoids false positives by understanding each model’s output format

Optimizing Performance: Use a Larger Context Window

Complex tool calls and multi-step conversations benefit from an expanded context window (number of tokens the model can “remember”).

How to set context size (cURL example):

curl -X POST "http://localhost:11434/api/chat" -d '{
  "model": "llama3.1",
  "messages": [
    {
      "role": "user",
      "content": "why is the sky blue?"
    }
  ],
  "options": {
    "num_ctx": 32000
  }
}'

• Larger num_ctx values (e.g., 32,000) allow the model to handle longer chats and more tool usage
• Check model support as not all models allow large contexts

Where to Go Next?

With Ollama’s streaming and tool call features, you can build:

• Real-time AI chatbots that interact with APIs and databases
• Automated agents for multi-step workflows
• Custom developer tools and monitoring bots

💡 For API developers, integrating Apidog into your workflow makes it easy to generate API documentation, run automated tests, and collaborate across your team. Apidog’s all-in-one platform is built for high-productivity API teams—see how it compares to Postman.