Models in PraisonAI

Overview of supported language models in PraisonAI, including OpenAI, Groq, Google Gemini, Anthropic Claude, and configuration examples

Point an Agent at any provider by setting its llm parameter — PraisonAI routes to OpenAI, Anthropic, Gemini, Groq, Cohere, or a local Ollama model.

from praisonaiagents import Agent

agent = Agent(instructions="You are a helpful assistant", llm="gpt-4o-mini")
agent.start("Why is the sky blue?")

Not sure which model to use? Run praisonai models list to browse all available models, or see the Model Catalogue CLI for full details on browsing, describing, and validating models.

Code

Set model by 3 ways

1. OpenAI Compatible Endpoints

export OPENAI_API_KEY="${GROQ_API_KEY:?Set GROQ_API_KEY in your shell}"
export OPENAI_BASE_URL=https://api.groq.com/openai/v1

from praisonaiagents import Agent

agent = Agent(
    instructions="You are a helpful assistant",
    llm="llama-3.1-8b-instant",
)

agent.start("Why sky is Blue?")

2. Litellm Compatible model names (eg: gemini/gemini-1.5-flash-8b)

pip install "praisonaiagents[llm]"

from praisonaiagents import Agent

agent = Agent(
    instructions="You are a helpful assistant",
    llm="gemini/gemini-1.5-flash-8b",
    reflection=True,
    
)

agent.start("Why sky is Blue?")

3. Litellm Compatible Configuration

pip install "praisonaiagents[llm]"

from praisonaiagents import Agent

llm_config = {
    "model": "gemini/gemini-1.5-flash-latest",  # Model name without provider prefix
    
    # Core settings
    "temperature": 0.7,                # Controls randomness (like temperature)
    "timeout": 30,                 # Timeout in seconds
    "top_p": 0.9,                    # Nucleus sampling parameter
    "max_tokens": 1000,               # Max tokens in response
    
    # Advanced parameters
    "presence_penalty": 0.1,         # Penalize repetition of topics (-2.0 to 2.0)
    "frequency_penalty": 0.1,        # Penalize token repetition (-2.0 to 2.0)
    
    # API settings (optional)
    "api_key": None,                 # Your API key (or use environment variable)
    "base_url": None,                # Custom API endpoint if needed
    
    # Response formatting
    "response_format": {             # Force specific response format
        "type": "text"               # Options: "text", "json_object"
    },
    
    # Additional controls
    "seed": 42,                      # For reproducible responses
    "stop_phrases": ["##", "END"],   # Custom stop sequences
}

agent = Agent(
    instructions="You are a helpful Assistant."
    llm=llm_config
)
agent.start()

Advanced Configuration (Litellm Support)

This uses Litellm

1. Install Package

   Install required packages:

   pip install "praisonaiagents[llm]"
   
2. Setup Environment

   Configure environment:

   export GOOGLE_API_KEY="${GOOGLE_API_KEY:?Set GOOGLE_API_KEY in your shell}"
   

   Get your API key from Google AI Studio

3. Create Agent

   Create app.py:

   Basic

   # if json_object is supported by the model
   from praisonaiagents import Agent
   
   agent = Agent(
       instructions="You are a helpful assistant",
       llm="gemini/gemini-1.5-flash-8b",
       reflection=True,
       
   )
   
   agent.start("Why sky is Blue?")
   

Ollama Integration

export OPENAI_BASE_URL=http://localhost:11434/v1

Groq Integration

export OPENAI_API_KEY="${GROQ_API_KEY:?Set GROQ_API_KEY in your shell}"
export OPENAI_BASE_URL=https://api.groq.com/openai/v1

Google Gemini

export OPENAI_API_KEY="${GEMINI_API_KEY:?Set GEMINI_API_KEY in your shell}"
export OPENAI_BASE_URL=https://generativelanguage.googleapis.com/v1beta/openai/

Jan AI Integration

export OPENAI_BASE_URL=http://localhost:1337/v1

LM Studio Integration

export OPENAI_BASE_URL=http://localhost:1234/v1

OpenRouter Integration

export OPENAI_API_KEY="${OPENROUTER_API_KEY:?Set OPENROUTER_API_KEY in your shell}"
export OPENAI_BASE_URL=https://openrouter.ai/api/v1

Provider Auto-Detection (no-config first run)

When you run praisonai run without setting --model or a model: key in config.yaml, PraisonAI inspects which supported provider credential is present in your environment and picks a provider-appropriate default — so a user whose only key is ANTHROPIC_API_KEY no longer gets an OpenAI auth error on first run.

Detection is catalogue-driven: any provider in PROVIDER_ENV_CATALOGUE (praisonaiagents/llm/catalogue.py) with its API-key env var set is recognised — not just the historical handful. The ordered preference is preserved (OpenAI still wins when multiple keys are set), and the terminal fallback is still gpt-4o-mini.

Credential env var(s)	Resolved default model
OPENAI_API_KEY	gpt-4o-mini
ANTHROPIC_API_KEY	anthropic/claude-3-5-sonnet-latest
GEMINI_API_KEY	gemini/gemini-1.5-flash
GOOGLE_API_KEY	google/gemini-1.5-flash
GROQ_API_KEY	groq/llama-3.3-70b-versatile
COHERE_API_KEY	cohere/command-r
OPENROUTER_API_KEY	openrouter/openai/gpt-4o-mini
OLLAMA_HOST	ollama/llama3.2
MISTRAL_API_KEY	mistral/mistral-large-latest
DEEPSEEK_API_KEY	deepseek/deepseek-chat
XAI_API_KEY	xai/grok-2-latest
TOGETHER_API_KEY / TOGETHERAI_API_KEY	together_ai/meta-llama/Llama-3.3-70B-Instruct-Turbo
PERPLEXITYAI_API_KEY	perplexity/sonar
FIREWORKS_API_KEY / FIREWORKS_AI_API_KEY	fireworks_ai/accounts/fireworks/models/llama-v3p3-70b-instruct
(none of the above)	gpt-4o-mini

Precedence: the first catalogued credential that is set wins, in the order above (the head of the catalogue preserves the historical preference). If multiple provider keys are set, the one listed first takes effect.

Setting llm="local" on an Agent bypasses this credential catalogue entirely and runs the Local Model Resolver — it discovers a running local server, picks the best model, and configures itself with no key. That is the explicit, user-driven local path; the keyless fallback below is the implicit one.

The same resolver drives implicit defaults for praisonai run, praisonai chat, praisonai code, praisonai init scaffolding, praisonai setup, and the bare-praisonai TUI launch — not just run. praisonai code now flows through the same resolver and first-run gate.

Same resolver from the Python SDK

Constructing Agent(instructions="…") without llm= uses the same _PROVIDER_DEFAULT_MODELS table — with OLLAMA_HOST set alone, the Agent picks ollama/llama3.2 and routes it through litellm.

import os
from praisonaiagents import Agent

os.environ["OLLAMA_HOST"] = "http://localhost:11434"

# No llm= argument, no OPENAI_API_KEY — the Agent picks ollama/llama3.2
# and routes through litellm.
agent = Agent(instructions="Answer briefly.")
agent.start("2+2")

Before PR #4795, six of the seven defaults resolved the right model name but were then routed to the native OpenAI client, causing ValueError: OPENAI_API_KEY environment variable is required. This is fixed as of #4795 — the SDK Agent() constructor now matches praisonai run. See Agent Model Resolution for the full flow.

Keyless local-first fallback (no env vars set)

If no cloud provider key is set and a local OpenAI-compatible endpoint answers on http://127.0.0.1:11434 (or wherever OPENAI_BASE_URL / OLLAMA_HOST points), PraisonAI uses that local server as the zero-config default — the first praisonai run "..." just works before you configure anything.

# Terminal 1
ollama serve
ollama pull llama3.2

# Terminal 2 — no OPENAI_API_KEY, no OLLAMA_HOST, nothing set
praisonai run "hello"
# > No cloud key found; using local model ollama/llama3.2.
# > Run `praisonai setup` to add a hosted provider.

Precedence (first match wins):

1. --model <name> on the command line.
2. Any cloud provider key from the table above.
3. A reachable local endpoint at OPENAI_BASE_URL → OLLAMA_HOST → http://127.0.0.1:11434.
4. gpt-4o-mini fallback.

Detection is timeout-bounded (~150 ms) and cached briefly, so the first-run hot path stays fast when nothing is listening. Cloud keys always win — the local probe is skipped entirely when any cloud key is set.

Detection recognises two shapes at the local server:

Server type	Probed path	Resolved model id
Ollama	GET /api/tags (at the root, not /v1/api/tags)	ollama/<first-model-name>
Generic OpenAI-compatible (llama.cpp, LM Studio, vLLM)	GET /v1/models	openai/<first-id> (routed through the local base URL)

See Local Models for the full guide.

This keyless fallback is the implicit local path. To request a local model explicitly — with full engine discovery and a loud failure when a named server is missing — set llm="local" on the Agent. See the Local Model Resolver.

An explicit --model <name> flag or a model: key in config.yaml always overrides auto-detection.

A local base_url no longer captures hosted model names — gpt-*, claude*, and gemini-* keep their native adapter even behind http://localhost:11434/v1. See Mixing local and hosted models.

No cloud key at all? If Ollama (or any OpenAI-compatible local server) is running, PraisonAI will use it — no API key and no config. See Keyless Local-First Run.

Routed model provider detection

When a model id routes through Bedrock, Vertex AI, or OpenRouter to reach Claude or Gemini, PraisonAI detects the true provider from the model id — not the routing prefix — so the correct streaming adapter is used.

Routed model id	Detected provider
bedrock/anthropic.claude-3-5-sonnet-v1:0	Anthropic
vertex_ai/claude-3-5-sonnet@20240620	Anthropic
openrouter/anthropic/claude-3.5-sonnet	Anthropic
vertex_ai/gemini-1.5-pro	Gemini
openrouter/google/gemini-1.5-pro	Gemini
gpt-4o, openai/...	OpenAI

Async streaming (agent.astart(...)) and agent.astream(...) now work correctly for these routed model ids. Older versions fell through to the OpenAI streaming adapter and raised a runtime error on the async path.

Detection first checks known provider prefixes, then falls back to substring matching: "claude" or "anthropic" in the model id resolves to Anthropic, and "gemini" resolves to Gemini. Genuinely-OpenAI ids (gpt-4o, openai/...) are unaffected.

Primary vs Auxiliary Model

PraisonAI splits your model choice into two knobs:

Knob	What it drives	Configured via
model	The agent’s own reasoning / tool-use turns	Agent(llm=…), [defaults].model, MODEL_NAME / OPENAI_MODEL_NAME env
small_model / auxiliary	Cheap internal calls — session titles, context compaction/summarisation, LLM guardrail validation, memory quality scoring, workflow routing, lite helpers	[defaults].small_model, or the PRAISONAI_AUXILIARY_MODEL env var

The auxiliary knob falls back to model when unset, so single-provider (Anthropic, Ollama, on-prem) setups make zero unexpected third-party calls — the auxiliary calls all stay on your configured provider.

Two config surfaces resolve the auxiliary model:

• Config-file resolver ([defaults].small_model) — used by session-title generation via get_small_model(primary_model, fallback).
• Env-var resolver (PRAISONAI_AUXILIARY_MODEL → OPENAI_MODEL_NAME → gpt-4o-mini) — used by memory quality scoring, context compaction, workflow routing, lite helpers, task callbacks, and learn-manager extraction. Session-title generation also falls through to this ladder when defaults.small_model is unset.

The env-var resolver covers 8 modules: memory/memory.py, memory/learn/manager.py, context/compressor.py, context/optimizer.py, workflows/workflows.py, lite/__init__.py, task/task.py, and session/title.py. Cost tables (utils/cost_utils.py, llm/_cost.py), docstring examples, and public dataclass field defaults are deliberately not routed — PRAISONAI_AUXILIARY_MODEL does not change pricing behaviour. Empty or whitespace-only env values (OPENAI_MODEL_NAME=" ") are ignored.

# .praisonai/config.toml
[defaults]
model = "gpt-4o"
small_model = "gpt-4o-mini"

# Or purely via environment — no config file required
export OPENAI_MODEL_NAME=ollama/llama3.3:70b        # primary agent model
export PRAISONAI_AUXILIARY_MODEL=ollama/qwen3:0.6b  # cheap helper model

See Configuration File → Cheap auxiliary model for internal calls for the full precedence ladder.

Supported Models for No Code

PraisonAI Chat	PraisonAI Code	PraisonAI (Multi-Agents)
Litellm	Litellm	Below Models

• OpenAI
• Groq
• Google Gemini
• Anthropic Claude
• Cohere
• Mistral
• Ollama
• Other Models

Example agents.yaml

This uses Multi-Agents with Multi-LLMs.

framework: crewai
topic: research about the causes of lung disease
agents:  # Canonical: use 'agents' instead of 'roles'
  research_analyst:
    instructions:  # Canonical: use 'instructions' instead of 'backstory' Experienced in analyzing scientific data related to respiratory health.
    goal: Analyze data on lung diseases
    role: Research Analyst
    llm:  
      model: "groq/llama3-70b-8192"
    function_calling_llm: 
      model: "google/gemini-1.5-flash-001"
    tasks:
      data_analysis:
        description: Gather and analyze data on the causes and risk factors of lung
          diseases.
        expected_output: Report detailing key findings on lung disease causes.
    tools:
    - 'InternetSearchTool'
  medical_writer:
    instructions:  # Canonical: use 'instructions' instead of 'backstory' Skilled in translating complex medical information into accessible
      content.
    goal: Compile comprehensive content on lung disease causes
    role: Medical Writer
    llm:  
      model: "anthropic/claude-3-haiku-20240307"
    function_calling_llm: 
      model: "openai/gpt-4o"
    tasks:
      content_creation:
        description: Create detailed content summarizing the research findings on
          lung disease causes.
        expected_output: Document outlining various causes and risk factors of lung
          diseases.
    tools:
    - ''
  editor:
    instructions:  # Canonical: use 'instructions' instead of 'backstory' Proficient in editing medical content for accuracy and clarity.
    goal: Review and refine content on lung disease causes
    role: Editor
    llm:  
      model: "cohere/command-r"
    tasks:
      content_review:
        description: Edit and refine the compiled content on lung disease causes for
          accuracy and coherence.
        expected_output: Finalized document on lung disease causes ready for dissemination.
    tools:
    - ''
dependencies: []

How It Works

The Agent passes your llm value to the provider resolver, which routes the request to the matching model and returns the response.

Best Practices

Let auto-detection pick the default

Skip llm= on first runs. PraisonAI resolves a sensible default from whichever provider key is set — see Provider Auto-Detection.

Use LiteLLM prefixes for non-OpenAI providers

Pass llm="gemini/gemini-1.5-flash-8b" or llm="anthropic/claude-3-5-sonnet-latest" to target a specific provider model.

Keep API keys in the environment

Set provider keys in your shell or .env. Use api_key=None in llm_config so the SDK reads the environment variable.

Match model to task

Use a small fast model (gpt-4o-mini, gemini-1.5-flash) for routing and a larger model only where quality matters.

Related

Quick Start

Run your first agent in a few lines.

Tools

Give models real actions with tools.

Local Model Resolver

Point an Agent at any running local server with llm="local".

Keyless Local-First Run

The implicit local fallback when no cloud key is set.