Ollama

Use local Ollama models with PraisonAI Agents

See also: ollama CLI for the praisonai ollama command with Weak-Model-Proof execution.

Using the desktop app? Enter the bare model name (llama3.2) plus a Base URL, not ollama/llama3.2. See Which models work in the desktop app.

Don’t want to hard-code ollama/<model> or the base URL? Use llm="local" — it discovers Ollama automatically and picks the best model. See the Local Model Resolver.

Common pitfall: naming the model matters

Setting only OPENAI_BASE_URL is not enough. With no model named, the OpenAI default gpt-4o-mini is sent to Ollama, which answers:

404 - {'error': {'message': "model 'gpt-4o-mini' not found", 'type': 'not_found_error'}}

Always name the model with the ollama/ prefix — it selects the OllamaAdapter (tool-call repair, tool-result formatting, small-model streaming rules) instead of the plain OpenAIClient.

python

from praisonaiagents import Agent

# Simplest — discovery finds Ollama and picks the model.
Agent(instructions="You are a helpful assistant", llm="local").start("Hello!")

# Or name the model yourself.
Agent(instructions="You are a helpful assistant", llm="ollama/llama3.2").start("Why is the sky blue?")

bash

export OPENAI_MODEL_NAME=ollama/llama3.2

As of PR #4799, the SDK emits a one-time warning per process when a non-OpenAI endpoint (via OPENAI_BASE_URL or OPENAI_API_BASE) is configured without a model. The warning names the endpoint, quotes the default that would be sent, and tells you how to fix it. It is a warning, not an error, because OpenAI-compatible proxies (LiteLLM, vLLM) may legitimately serve gpt-4o-mini under that name.

Custom Ollama host

If Ollama runs on a non-default host or port, either pass base_url= or set OLLAMA_HOST:

from praisonaiagents import Agent

agent = Agent(
    instructions="You are a helpful assistant",
    llm="ollama/llama3.2",
    base_url="http://192.168.1.10:11434",
)
agent.start("Why is the sky blue?")

Since PR #4870, a path prefix on OLLAMA_HOST survives end-to-end: OLLAMA_HOST=http://gateway:8000/ollama keeps the /ollama prefix. Before that, the prefix was silently dropped and every request went to the proxy root.

Other local runtimes

The same provider/model shape works for other local runtimes:

Runtime	Prefix	Notes
Ollama	ollama/	Tool-call repair via OllamaAdapter
Ollama (tool calling)	ollama_chat/	Recommended for tool-using models
LM Studio	lm_studio/	Point base_url at the LM Studio server
vLLM	vllm/ or hosted_vllm/	Depending on server mode

Diagnostic warning

When a non-OpenAI endpoint is configured via OPENAI_BASE_URL or OPENAI_API_BASE and no model is named, the SDK emits this one-time warning per process:

No model specified, so the OpenAI default 'gpt-4o-mini' will be sent to
http://localhost:11434/v1. That endpoint probably does not serve it. Set
OPENAI_MODEL_NAME to a model it does serve (e.g. OPENAI_MODEL_NAME=ollama/llama3.2),
or pass llm= explicitly.

Silence it by naming a model. The warning does not fire for real OpenAI, for an explicit llm=, or when a provider credential (e.g. OLLAMA_HOST) already resolves a prefixed default. It fires once per process, so building many agents in a loop does not repeat it.

Environment Variables

Setting MODEL_NAME=ollama/llama3.2 is enough — the default base URL http://localhost:11434/v1 is used automatically. The OLLAMA_API_KEY environment variable is consulted; OPENAI_API_KEY is not used as a fallback (though most local Ollama setups don’t need an API key).

If all you want is Ollama’s default llama3.2, setting OLLAMA_HOST alone is enough — no MODEL_NAME, no llm=. Agent(...) picks ollama/llama3.2 and routes it through litellm. See Provider Auto-Detection.

import os
from praisonaiagents import Agent

os.environ["OLLAMA_HOST"] = "http://localhost:11434"

# No MODEL_NAME, no llm= — the Agent picks ollama/llama3.2.
agent = Agent(instructions="You run locally via Ollama")
agent.start("2+2")

import os
from praisonaiagents import Agent

os.environ["MODEL_NAME"] = "ollama/llama3.2"
# API key usually not needed for local Ollama

agent = Agent(name="Local Agent", instructions="You run locally via Ollama")

Direct provider-registry route

Since PR #4800, the ollama and ollama_chat prefixes are registered providers — you can build a provider directly, no environment variables needed:

from praisonai.llm import create_llm_provider

provider = create_llm_provider("ollama/llama3.2")
# or, for tool-calling models:
provider = create_llm_provider("ollama_chat/llama3.2")

Use ollama_chat/… when the model needs to call tools — it is LiteLLM’s recommended prefix for Ollama tool calling. Everything else keeps using ollama/….

Tool-calling models: use ollama_chat/…

For any Ollama model you plan to use with tools, prefer the ollama_chat/… prefix over the bare ollama/…. Both hit the same Ollama server, but ollama_chat/… is LiteLLM’s recommended spelling for tool calling and — since PR #4798 — PraisonAI routes it through the same OllamaAdapter used by ollama/…, with max_tool_repairs=2. A small local model that emits a malformed tool call now gets two automatic repair attempts instead of failing the conversation.

from praisonaiagents import Agent

def get_weather(city: str) -> str:
    return f"{city}: 21°C"

agent = Agent(
    name="Local Assistant",
    instructions="Answer using the tool when relevant.",
    llm="ollama_chat/qwen3:0.6b",   # tool-calling prefix
    tools=[get_weather],
)
agent.start("What is the weather in Paris?")

No environment variables or manual adapter config required — the prefix alone selects the right adapter.

Argument filtering on every response path

When a small local model emits tool call arguments that don’t belong to the tool’s signature (a common failure mode of weak Ollama models), PraisonAI drops the extra arguments before dispatch on all three response paths — sync, async, and streaming. PR #4810 closed the last streaming-path gap, so a streamed tool call is now filtered the same way sync and async already were. The filter is a no-op for every other provider.

from praisonaiagents import Agent

def get_weather(city: str) -> str:
    return f"{city}: 21°C"

agent = Agent(
    name="Local Assistant",
    instructions="Answer using the tool when relevant.",
    llm="ollama_chat/qwen3:0.6b",
    tools=[get_weather],
    stream=True,  # streaming is now protected too
)
# If the model emits {"city": "Paris", "stock_symbol": "AAPL"},
# the tool is called with just {"city": "Paris"} — the stray argument is dropped.
agent.start("What is the weather in Paris?")

Tool calls emitted as plain text

Some local models answer a tool call as JSON in the response body instead of using the tool-call field. OllamaAdapter recovers that JSON and dispatches the tool; other providers (OpenAI, Anthropic, Gemini) do not — they treat a JSON-looking answer as plain text. See OpenAI-Compatible → Local prefix route for how this compares across adapters.

How Ollama is handled differently

Ollama’s protocol has known quirks. The OllamaAdapter compensates so your agent code doesn’t have to know about any of them.

Ollama behaviour	What the adapter does
Returns empty content after a tool call	Detects it and prompts the model for a final answer instead of ending the turn
Doesn’t reliably stream tool calls	Disables streaming-with-tools for Ollama; falls back to non-streaming for that path
Small models emit tool calls as JSON text	Parses the JSON and dispatches the tool call anyway (up to 2 repair attempts)
Sends unsupported tool arguments during streaming	Strips them before the tool runs
Tool results delivered as role: "tool" confuse small models	Rewrites the result as a role: "user" natural-language turn so the model reads it correctly
Tool returned an error	Rewrites as an apology-oriented instruction, not the raw error string

All of this fires automatically when the llm= string routes to the OllamaAdapter (any ollama/…, ollama_chat/…, or the local auto-discovery when it finds Ollama).

ollama_chat/… is now treated as Ollama by every internal code path — streaming, tool summarisation, and empty-response handling — not just by adapter selection. Before PR #4823, the internal predicate matched only ollama/…, so ollama_chat/… could behave subtly differently even though it already routed to the OllamaAdapter. Now every path agrees.

Small tool-capable model (verified)

For CI, demos, and slower machines, qwen3:0.6b is the smallest Ollama model verified to complete a tool round-trip reliably.

from praisonaiagents import Agent

def add(a: int, b: int) -> int:
    """Add two integers."""
    return a + b

agent = Agent(
    name="Calculator",
    llm={
        "model": "ollama/qwen3:0.6b",
        "force_tool_usage": "always",
        "temperature": 0,
    },
    tools=[add],
)
agent.chat("Compute 17 + 25. You MUST use the calculator tool.")

It is 522 MB and carries the tools capability, unlike some sub-1B alternatives (smollm2:135m has no tools capability and cannot call tools at all).

Measured behaviour against a real Ollama server (PR #4803):

Configuration	Trials	Tool invoked + correct
Default	4	3 / 4
force_tool_usage="always" + temperature=0	6	6 / 6

The 6/6 configuration is the same one the CI contract test uses.

force_tool_usage and max_tool_repairs are llm dict entries, not top-level Agent(...) kwargs. Passing them as Agent(..., force_tool_usage="always") raises TypeError.

Multi-step arithmetic (e.g. (17+25)+(8+9)) at this size is unreliable — 0/2 in the same measurements, one trial hung > 90 s. For chained tool calls, prefer a larger model or split the request.

Models

Run models locally with Ollama. Popular options:

• Recommended: ollama/llama3.2 (latest Llama)
• Reasoning: ollama/deepseek-r1 (reasoning model)
• Small: ollama/qwen3 (efficient)
• Code: ollama/codellama (coding tasks)
• Tool calling: ollama_chat/qwen3 (LiteLLM-recommended prefix; auto-repair on)
• Tool calling (small, verified): ollama/qwen3:0.6b (522 MB; use force_tool_usage="always" + temperature=0 — see “Small tool-capable model” above)

Setup

# Install Ollama
curl -fsSL https://ollama.com/install.sh | sh

# Start Ollama server
ollama serve

# Pull a model
ollama pull llama3.2

For inference (running agents against an Ollama model), keep ollama serve running in a terminal. For training/pushing via praisonai train, the daemon is started automatically — no manual ollama serve needed.

Python

# No API key needed - runs locally
from praisonaiagents import Agent

agent = Agent(
    instructions="You are a helpful assistant",
    llm="ollama/llama3.2"
)
agent.start("Explain deep learning")

With Tools

from praisonaiagents import Agent

def read_file(path: str) -> str:
    """Read a file's contents."""
    with open(path, 'r') as f:
        return f.read()

agent = Agent(
    instructions="You are a code assistant",
    llm="ollama/codellama",
    tools=[read_file]
)
agent.start("Read and explain main.py")

Multi-Agent

from praisonaiagents import Agent, Task, AgentTeam

researcher = Agent(
    instructions="You research topics thoroughly",
    llm="ollama/llama3.2"
)
writer = Agent(
    instructions="You write clear summaries",
    llm="ollama/qwen3"
)

task1 = Task(description="Research Python best practices", agent=researcher)
task2 = Task(description="Write a guide", agent=writer)

agents = AgentTeam(agents=[researcher, writer], tasks=[task1, task2])
agents.start()

DeepSeek Reasoning

from praisonaiagents import Agent

agent = Agent(
    instructions="You are a problem solver",
    llm="ollama/deepseek-r1"
)
agent.start("Solve this math problem: What is 15% of 240?")

CLI

# Basic prompt
python -m praisonai "Explain AI" --ollama llama3.2

# With specific model
python -m praisonai "Write code" --llm ollama/codellama

# Run agents.yaml
python -m praisonai

YAML

framework: praisonai
topic: Local AI development
agents:
  coder:
    role: Software Developer
    goal: Write clean code
    instructions: You are an expert programmer
    llm:
      model: ollama/codellama
    tasks:
      code_task:
        description: Write a Python function to sort a list
        expected_output: Clean, documented Python code

  reviewer:
    role: Code Reviewer
    goal: Review and improve code
    instructions: You review code for best practices
    llm:
      model: ollama/llama3.2
    tasks:
      review_task:
        description: Review the code and suggest improvements
        expected_output: Code review with suggestions