Rediger

Del via

Ollama

Ollama allows you to run open-source models locally and use them with Agent Framework. This is ideal for development, testing, and scenarios where you need to keep data on-premises.

The following example shows how to create an agent using Ollama:

using System;
using Microsoft.Agents.AI;
using Microsoft.Extensions.AI;

// Create an Ollama agent using Microsoft.Extensions.AI.Ollama
// Requires: dotnet add package Microsoft.Extensions.AI.Ollama --prerelease
var chatClient = new OllamaChatClient(
    new Uri("http://localhost:11434"),
    modelId: "llama3.2");

AIAgent agent = chatClient.AsAIAgent(
    instructions: "You are a helpful assistant running locally via Ollama.");

Console.WriteLine(await agent.RunAsync("What is the largest city in France?"));

Prerequisites

Ensure Ollama is installed and running locally with a model downloaded before running any examples:

ollama pull llama3.2

Note

Not all models support function calling. For tool usage, try llama3.2 or qwen3:4b.

Installation

pip install agent-framework-ollama --pre

Configuration

OLLAMA_MODEL_ID="llama3.2"

The native client connects to http://localhost:11434 by default. You can override this by passing host to the client.

Create Ollama Agents

OllamaChatClient provides native Ollama integration with full support for function tools and streaming.

import asyncio
from agent_framework.ollama import OllamaChatClient

async def main():
    agent = OllamaChatClient().as_agent(
        name="HelpfulAssistant",
        instructions="You are a helpful assistant running locally via Ollama.",
    )
    result = await agent.run("What is the largest city in France?")
    print(result)

asyncio.run(main())

Function Tools

import asyncio
from datetime import datetime
from agent_framework.ollama import OllamaChatClient

def get_time(location: str) -> str:
    """Get the current time."""
    return f"The current time in {location} is {datetime.now().strftime('%I:%M %p')}."

async def main():
    agent = OllamaChatClient().as_agent(
        name="TimeAgent",
        instructions="You are a helpful time agent.",
        tools=get_time,
    )
    result = await agent.run("What time is it in Seattle?")
    print(result)

asyncio.run(main())

Streaming

async def streaming_example():
    agent = OllamaChatClient().as_agent(
        instructions="You are a helpful assistant.",
    )
    print("Agent: ", end="", flush=True)
    async for chunk in agent.run("Tell me about Python.", stream=True):
        if chunk.text:
            print(chunk.text, end="", flush=True)
    print()

Next steps

Providers Overview

  • Last updated on