Local Models

Run PraisonAI against Ollama or any OpenAI-compatible local server — keyless

Point PraisonAI at Ollama, LM Studio, llama.cpp, or vLLM — no cloud key required.

from praisonaiagents import Agent

agent = Agent(
    name="Local Assistant",
    instructions="Answer briefly",
    llm="ollama/llama3.2",
)
agent.start("Why is the sky blue?")

Quick Start

1. Zero-config with Ollama

   Start Ollama, pull a model, and run — PraisonAI detects the endpoint when no cloud key is set.

   ollama serve
   ollama pull llama3.2
   praisonai run "hello"
   # > No cloud key found; using local model ollama/llama3.2.
   
2. Point at any OpenAI-compatible server

   Set OPENAI_BASE_URL to target LM Studio, llama.cpp, or vLLM.

   export OPENAI_BASE_URL=http://localhost:1234/v1
   praisonai run "hello"
   

---

How It Works

PraisonAI probes for a reachable local server only when no cloud provider key is set, then adopts the first model it finds.

Server type	Probed path	Resolved model id
Ollama	GET /api/tags (at the root, not /v1/api/tags)	ollama/<first-model-name>
Generic OpenAI-compatible (llama.cpp, LM Studio, vLLM)	GET /v1/models	openai/<first-id>

Detection is timeout-bounded (~150 ms) and caches negatives for 30 s, so the first-run hot path stays fast when nothing is listening.

---

Configuration Options

Env var	Meaning	Default
OPENAI_BASE_URL	Any OpenAI-compatible base URL (with or without /v1)	—
OLLAMA_HOST	Ollama root (host:port allowed — a scheme is added if missing)	—
(none set)	Probe http://127.0.0.1:11434	—
PRAISONAI_HOME	Where session / credential data lives	~/.praisonai

Precedence (first match wins): --model <name> → any cloud provider key → reachable local endpoint (OPENAI_BASE_URL → OLLAMA_HOST → 127.0.0.1:11434) → gpt-4o-mini fallback.

---

Common Patterns

Ollama

ollama serve
ollama pull llama3.2
praisonai run "hello"

export OPENAI_BASE_URL=http://localhost:1234/v1
praisonai run "hello"

export OPENAI_BASE_URL=http://localhost:8080/v1
praisonai run "hello"

export OPENAI_BASE_URL=http://localhost:8000/v1
praisonai run "hello"

Pin a specific model explicitly with --model:

praisonai run "hello" --model ollama/llama3.2

---

Best Practices

Detection stays off the hot path

The probe runs only when no cloud key is set, times out at ~150 ms, and caches negatives for 30 s — a first run with nothing listening is not slowed down.

Cloud keys always win

Any cloud provider key (e.g. OPENAI_API_KEY, ANTHROPIC_API_KEY) skips the local probe entirely. OLLAMA_HOST is not a cloud key, so it participates in local detection instead.

Use --model to override detection

Pass --model ollama/llama3.2 (or any id) to bypass auto-detection and target an exact model.

Relocate state with PRAISONAI_HOME

Set PRAISONAI_HOME to move sessions, credentials, and cache in one place — handy for Nix, Docker, or Snap packaging. Defaults to ~/.praisonai.

---

Related

Provider Auto-Detection

The full precedence reference for model resolution.

First-run Onboarding

The complete credential resolution ladder.