> ## Documentation Index
> Fetch the complete documentation index at: https://praison.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Ollama

> Use local Ollama models with PraisonAI Agents

<Note>
  See also: [ollama CLI](/docs/cli/ollama) for the `praisonai ollama` command with Weak-Model-Proof execution.
</Note>

<Note>
  Using the **desktop app**? Enter the bare model name (`llama3.2`) plus a Base URL, not `ollama/llama3.2`. See [Which models work in the desktop app](/docs/install/desktop#which-models-work-in-the-desktop-app).
</Note>

<Tip>
  Don't want to hard-code `ollama/<model>` or the base URL? Use `llm="local"` — it discovers Ollama automatically and picks the best model. See the [Local Model Resolver](/docs/features/local-model-resolver).
</Tip>

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
graph LR
    subgraph "Naming the model matters"
        A[📝 llm=ollama/llama3.2] --> B[🔧 OllamaAdapter]
        B --> C[✅ Tool-call repair]
        D[📝 OPENAI_BASE_URL only] --> E[🔧 gpt-4o-mini sent]
        E --> F[❌ 404 not found]
    end

    classDef good fill:#10B981,stroke:#7C90A0,color:#fff
    classDef adapter fill:#189AB4,stroke:#7C90A0,color:#fff
    classDef input fill:#8B0000,stroke:#7C90A0,color:#fff
    classDef bad fill:#F59E0B,stroke:#7C90A0,color:#fff

    class A,D input
    class B adapter
    class C good
    class E bad
    class F bad
```

## Common pitfall: naming the model matters

Setting only `OPENAI_BASE_URL` is not enough. With no model named, the OpenAI default `gpt-4o-mini` is sent to Ollama, which answers:

```
404 - {'error': {'message': "model 'gpt-4o-mini' not found", 'type': 'not_found_error'}}
```

Always name the model with the `ollama/` prefix — it selects the `OllamaAdapter` (tool-call repair, tool-result formatting, small-model streaming rules) instead of the plain `OpenAIClient`.

<Tabs>
  <Tab title="python">
    ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    from praisonaiagents import Agent

    # Simplest — discovery finds Ollama and picks the model.
    Agent(instructions="You are a helpful assistant", llm="local").start("Hello!")

    # Or name the model yourself.
    Agent(instructions="You are a helpful assistant", llm="ollama/llama3.2").start("Why is the sky blue?")
    ```
  </Tab>

  <Tab title="bash">
    ```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    export OPENAI_MODEL_NAME=ollama/llama3.2
    ```
  </Tab>
</Tabs>

<Note>
  As of [PR #4799](https://github.com/MervinPraison/PraisonAI/pull/4799), the SDK emits a one-time warning per process when a non-OpenAI endpoint (via `OPENAI_BASE_URL` or `OPENAI_API_BASE`) is configured without a model. The warning names the endpoint, quotes the default that would be sent, and tells you how to fix it. It is a warning, not an error, because OpenAI-compatible proxies (LiteLLM, vLLM) may legitimately serve `gpt-4o-mini` under that name.
</Note>

## Custom Ollama host

If Ollama runs on a non-default host or port, either pass `base_url=` or set `OLLAMA_HOST`:

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents import Agent

agent = Agent(
    instructions="You are a helpful assistant",
    llm="ollama/llama3.2",
    base_url="http://192.168.1.10:11434",
)
agent.start("Why is the sky blue?")
```

<Note>
  Since PR [#4870](https://github.com/MervinPraison/PraisonAI/pull/4870), a path prefix on `OLLAMA_HOST` survives end-to-end: `OLLAMA_HOST=http://gateway:8000/ollama` keeps the `/ollama` prefix. Before that, the prefix was silently dropped and every request went to the proxy root.
</Note>

## Other local runtimes

The same `provider/model` shape works for other local runtimes:

| Runtime               | Prefix                    | Notes                                    |
| --------------------- | ------------------------- | ---------------------------------------- |
| Ollama                | `ollama/`                 | Tool-call repair via `OllamaAdapter`     |
| Ollama (tool calling) | `ollama_chat/`            | Recommended for tool-using models        |
| LM Studio             | `lm_studio/`              | Point `base_url` at the LM Studio server |
| vLLM                  | `vllm/` or `hosted_vllm/` | Depending on server mode                 |

## Diagnostic warning

When a non-OpenAI endpoint is configured via `OPENAI_BASE_URL` or `OPENAI_API_BASE` and no model is named, the SDK emits this one-time warning per process:

```
No model specified, so the OpenAI default 'gpt-4o-mini' will be sent to
http://localhost:11434/v1. That endpoint probably does not serve it. Set
OPENAI_MODEL_NAME to a model it does serve (e.g. OPENAI_MODEL_NAME=ollama/llama3.2),
or pass llm= explicitly.
```

Silence it by naming a model. The warning does **not** fire for real OpenAI, for an explicit `llm=`, or when a provider credential (e.g. `OLLAMA_HOST`) already resolves a prefixed default. It fires once per process, so building many agents in a loop does not repeat it.

## Environment Variables

Setting `MODEL_NAME=ollama/llama3.2` is enough — the default base URL `http://localhost:11434/v1` is used automatically. The `OLLAMA_API_KEY` environment variable is consulted; `OPENAI_API_KEY` is **not** used as a fallback (though most local Ollama setups don't need an API key).

If **all you want** is Ollama's default `llama3.2`, setting `OLLAMA_HOST` alone is enough — no `MODEL_NAME`, no `llm=`. `Agent(...)` picks `ollama/llama3.2` and routes it through litellm. See [Provider Auto-Detection](/docs/models#provider-auto-detection-no-config-first-run).

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
import os
from praisonaiagents import Agent

os.environ["OLLAMA_HOST"] = "http://localhost:11434"

# No MODEL_NAME, no llm= — the Agent picks ollama/llama3.2.
agent = Agent(instructions="You run locally via Ollama")
agent.start("2+2")
```

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
import os
from praisonaiagents import Agent

os.environ["MODEL_NAME"] = "ollama/llama3.2"
# API key usually not needed for local Ollama

agent = Agent(name="Local Agent", instructions="You run locally via Ollama")
```

## Direct provider-registry route

Since [PR #4800](https://github.com/MervinPraison/PraisonAI/pull/4800), the `ollama` and `ollama_chat` prefixes are registered providers — you can build a provider directly, no environment variables needed:

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonai.llm import create_llm_provider

provider = create_llm_provider("ollama/llama3.2")
# or, for tool-calling models:
provider = create_llm_provider("ollama_chat/llama3.2")
```

Use `ollama_chat/…` when the model needs to call tools — it is LiteLLM's recommended prefix for Ollama tool calling. Everything else keeps using `ollama/…`.

### Tool-calling models: use `ollama_chat/…`

For any Ollama model you plan to use with tools, prefer the `ollama_chat/…` prefix over the bare `ollama/…`. Both hit the same Ollama server, but `ollama_chat/…` is LiteLLM's recommended spelling for tool calling and — since [PR #4798](https://github.com/MervinPraison/PraisonAI/pull/4798) — PraisonAI routes it through the same `OllamaAdapter` used by `ollama/…`, with `max_tool_repairs=2`. A small local model that emits a malformed tool call now gets two automatic repair attempts instead of failing the conversation.

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents import Agent

def get_weather(city: str) -> str:
    return f"{city}: 21°C"

agent = Agent(
    name="Local Assistant",
    instructions="Answer using the tool when relevant.",
    llm="ollama_chat/qwen3:0.6b",   # tool-calling prefix
    tools=[get_weather],
)
agent.start("What is the weather in Paris?")
```

No environment variables or manual adapter config required — the prefix alone selects the right adapter.

### Argument filtering on every response path

When a small local model emits tool call arguments that don't belong to the tool's signature (a common failure mode of weak Ollama models), PraisonAI drops the extra arguments before dispatch on **all three** response paths — sync, async, and streaming. [PR #4810](https://github.com/MervinPraison/PraisonAI/pull/4810) closed the last streaming-path gap, so a streamed tool call is now filtered the same way sync and async already were. The filter is a no-op for every other provider.

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents import Agent

def get_weather(city: str) -> str:
    return f"{city}: 21°C"

agent = Agent(
    name="Local Assistant",
    instructions="Answer using the tool when relevant.",
    llm="ollama_chat/qwen3:0.6b",
    tools=[get_weather],
    stream=True,  # streaming is now protected too
)
# If the model emits {"city": "Paris", "stock_symbol": "AAPL"},
# the tool is called with just {"city": "Paris"} — the stray argument is dropped.
agent.start("What is the weather in Paris?")
```

### Tool calls emitted as plain text

Some local models answer a tool call as JSON in the response body instead of using the tool-call field. `OllamaAdapter` recovers that JSON and dispatches the tool; other providers (OpenAI, Anthropic, Gemini) do not — they treat a JSON-looking answer as plain text. See [OpenAI-Compatible → Local prefix route](/docs/models/openai-compatible#local-prefix-route-tool-call-repair) for how this compares across adapters.

## How Ollama is handled differently

Ollama's protocol has known quirks. The `OllamaAdapter` compensates so your agent code doesn't have to know about any of them.

| Ollama behaviour                                              | What the adapter does                                                                         |
| ------------------------------------------------------------- | --------------------------------------------------------------------------------------------- |
| Returns empty content after a tool call                       | Detects it and prompts the model for a final answer instead of ending the turn                |
| Doesn't reliably stream tool calls                            | Disables streaming-with-tools for Ollama; falls back to non-streaming for that path           |
| Small models emit tool calls as JSON text                     | Parses the JSON and dispatches the tool call anyway (up to 2 repair attempts)                 |
| Sends unsupported tool arguments during streaming             | Strips them before the tool runs                                                              |
| Tool results delivered as `role: "tool"` confuse small models | Rewrites the result as a `role: "user"` natural-language turn so the model reads it correctly |
| Tool returned an error                                        | Rewrites as an apology-oriented instruction, not the raw error string                         |

All of this fires automatically when the `llm=` string routes to the `OllamaAdapter` (any `ollama/…`, `ollama_chat/…`, or the `local` auto-discovery when it finds Ollama).

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
graph TB
    Route[📝 llm='ollama_chat/…'] --> Adapter[🔌 OllamaAdapter]
    Adapter --> Empty[🔁 Empty-response prompt]
    Adapter --> NoStream[⏭️ No stream with tools]
    Adapter --> Recover[🧩 JSON-text recovery]
    Adapter --> ArgFilter[🔎 Stream arg filter]
    Adapter --> Rewrite[💬 role:user rewrite]
    Adapter --> ErrRewrite[🙇 Apology-oriented error]

    classDef input fill:#8B0000,stroke:#7C90A0,color:#fff
    classDef adapter fill:#189AB4,stroke:#7C90A0,color:#fff
    classDef fix fill:#F59E0B,stroke:#7C90A0,color:#fff

    class Route input
    class Adapter adapter
    class Empty,NoStream,Recover,ArgFilter,Rewrite,ErrRewrite fix
```

<Note>
  `ollama_chat/…` is now treated as Ollama by **every** internal code path — streaming, tool summarisation, and empty-response handling — not just by adapter selection. Before [PR #4823](https://github.com/MervinPraison/PraisonAI/pull/4823), the internal predicate matched only `ollama/…`, so `ollama_chat/…` could behave subtly differently even though it already routed to the `OllamaAdapter`. Now every path agrees.
</Note>

### Small tool-capable model (verified)

For CI, demos, and slower machines, `qwen3:0.6b` is the smallest Ollama model verified to complete a tool round-trip reliably.

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents import Agent

def add(a: int, b: int) -> int:
    """Add two integers."""
    return a + b

agent = Agent(
    name="Calculator",
    llm={
        "model": "ollama/qwen3:0.6b",
        "force_tool_usage": "always",
        "temperature": 0,
    },
    tools=[add],
)
agent.chat("Compute 17 + 25. You MUST use the calculator tool.")
```

It is **522 MB** and carries the `tools` capability, unlike some sub-1B alternatives (`smollm2:135m` has no `tools` capability and cannot call tools at all).

Measured behaviour against a real Ollama server ([PR #4803](https://github.com/MervinPraison/PraisonAI/pull/4803)):

| Configuration                                 | Trials | Tool invoked + correct |
| --------------------------------------------- | ------ | ---------------------- |
| Default                                       | 4      | 3 / 4                  |
| `force_tool_usage="always"` + `temperature=0` | 6      | **6 / 6**              |

The 6/6 configuration is the same one the CI contract test uses.

<Note>
  `force_tool_usage` and `max_tool_repairs` are **`llm` dict entries**, not top-level `Agent(...)` kwargs. Passing them as `Agent(..., force_tool_usage="always")` raises `TypeError`.
</Note>

Multi-step arithmetic (e.g. `(17+25)+(8+9)`) at this size is unreliable — 0/2 in the same measurements, one trial hung > 90 s. For chained tool calls, prefer a larger model or split the request.

## Models

Run models locally with Ollama. Popular options:

* **Recommended**: `ollama/llama3.2` (latest Llama)
* **Reasoning**: `ollama/deepseek-r1` (reasoning model)
* **Small**: `ollama/qwen3` (efficient)
* **Code**: `ollama/codellama` (coding tasks)
* **Tool calling**: `ollama_chat/qwen3` (LiteLLM-recommended prefix; auto-repair on)
* **Tool calling (small, verified)**: `ollama/qwen3:0.6b` (522 MB; use `force_tool_usage="always"` + `temperature=0` — see "Small tool-capable model" above)

## Setup

```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
# Install Ollama
curl -fsSL https://ollama.com/install.sh | sh

# Start Ollama server
ollama serve

# Pull a model
ollama pull llama3.2
```

<Tip>
  For inference (running agents against an Ollama model), keep `ollama serve` running in a terminal. For **training/pushing** via `praisonai train`, the daemon is started automatically — no manual `ollama serve` needed.
</Tip>

## Python

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
# No API key needed - runs locally
from praisonaiagents import Agent

agent = Agent(
    instructions="You are a helpful assistant",
    llm="ollama/llama3.2"
)
agent.start("Explain deep learning")
```

### With Tools

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents import Agent

def read_file(path: str) -> str:
    """Read a file's contents."""
    with open(path, 'r') as f:
        return f.read()

agent = Agent(
    instructions="You are a code assistant",
    llm="ollama/codellama",
    tools=[read_file]
)
agent.start("Read and explain main.py")
```

### Multi-Agent

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents import Agent, Task, AgentTeam

researcher = Agent(
    instructions="You research topics thoroughly",
    llm="ollama/llama3.2"
)
writer = Agent(
    instructions="You write clear summaries",
    llm="ollama/qwen3"
)

task1 = Task(description="Research Python best practices", agent=researcher)
task2 = Task(description="Write a guide", agent=writer)

agents = AgentTeam(agents=[researcher, writer], tasks=[task1, task2])
agents.start()
```

### DeepSeek Reasoning

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents import Agent

agent = Agent(
    instructions="You are a problem solver",
    llm="ollama/deepseek-r1"
)
agent.start("Solve this math problem: What is 15% of 240?")
```

## CLI

```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
# Basic prompt
python -m praisonai "Explain AI" --ollama llama3.2

# With specific model
python -m praisonai "Write code" --llm ollama/codellama

# Run agents.yaml
python -m praisonai
```

## YAML

```yaml theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
framework: praisonai
topic: Local AI development
agents:
  coder:
    role: Software Developer
    goal: Write clean code
    instructions: You are an expert programmer
    llm:
      model: ollama/codellama
    tasks:
      code_task:
        description: Write a Python function to sort a list
        expected_output: Clean, documented Python code

  reviewer:
    role: Code Reviewer
    goal: Review and improve code
    instructions: You review code for best practices
    llm:
      model: ollama/llama3.2
    tasks:
      review_task:
        description: Review the code and suggest improvements
        expected_output: Code review with suggestions
```