> ## Documentation Index > Fetch the complete documentation index at: https://praison.ai/docs/llms.txt > Use this file to discover all available pages before exploring further. # Models in PraisonAI > Overview of supported language models in PraisonAI, including OpenAI, Groq, Google Gemini, Anthropic Claude, and configuration examples Point an Agent at any provider by setting its `llm` parameter — PraisonAI routes to OpenAI, Anthropic, Gemini, Groq, Cohere, or a local Ollama model. ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} from praisonaiagents import Agent agent = Agent(instructions="You are a helpful assistant", llm="gpt-4o-mini") agent.start("Why is the sky blue?") ``` ```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} graph LR subgraph "Model Selection" User[📋 User] --> Agent[🤖 Agent] Agent --> Router[🧠 Provider Resolver] Router --> Model[✅ LLM Response] end classDef input fill:#8B0000,stroke:#7C90A0,color:#fff classDef process fill:#189AB4,stroke:#7C90A0,color:#fff classDef output fill:#10B981,stroke:#7C90A0,color:#fff class User,Agent input class Router process class Model output ``` Not sure which model to use? Run `praisonai models list` to browse all available models, or see the [Model Catalogue CLI](/docs/features/models-cli) for full details on browsing, describing, and validating models. # Code ## Set model by 3 ways ### 1. OpenAI Compatible Endpoints By Default it uses OPENAI\_BASE\_URL [https://api.openai.com/v1](https://api.openai.com/v1) Example Groq Implementation: ```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} export OPENAI_API_KEY="${GROQ_API_KEY:?Set GROQ_API_KEY in your shell}" export OPENAI_BASE_URL=https://api.groq.com/openai/v1 ``` ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} from praisonaiagents import Agent agent = Agent( instructions="You are a helpful assistant", llm="llama-3.1-8b-instant", ) agent.start("Why sky is Blue?") ``` ### 2. Litellm Compatible model names (eg: gemini/gemini-1.5-flash-8b) ```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} pip install "praisonaiagents[llm]" ``` ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} from praisonaiagents import Agent agent = Agent( instructions="You are a helpful assistant", llm="gemini/gemini-1.5-flash-8b", reflection=True, ) agent.start("Why sky is Blue?") ``` ### 3. Litellm Compatible Configuration ```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} pip install "praisonaiagents[llm]" ``` ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} from praisonaiagents import Agent llm_config = { "model": "gemini/gemini-1.5-flash-latest", # Model name without provider prefix # Core settings "temperature": 0.7, # Controls randomness (like temperature) "timeout": 30, # Timeout in seconds "top_p": 0.9, # Nucleus sampling parameter "max_tokens": 1000, # Max tokens in response # Advanced parameters "presence_penalty": 0.1, # Penalize repetition of topics (-2.0 to 2.0) "frequency_penalty": 0.1, # Penalize token repetition (-2.0 to 2.0) # API settings (optional) "api_key": None, # Your API key (or use environment variable) "base_url": None, # Custom API endpoint if needed # Response formatting "response_format": { # Force specific response format "type": "text" # Options: "text", "json_object" }, # Additional controls "seed": 42, # For reproducible responses "stop_phrases": ["##", "END"], # Custom stop sequences } agent = Agent( instructions="You are a helpful Assistant." llm=llm_config ) agent.start() ``` ## Advanced Configuration (Litellm Support) This uses Litellm Install required packages: ```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} pip install "praisonaiagents[llm]" ``` Configure environment: ```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} export GOOGLE_API_KEY="${GOOGLE_API_KEY:?Set GOOGLE_API_KEY in your shell}" ``` Get your API key from [Google AI Studio](https://makersuite.google.com/app/apikey) Create `app.py`: ```python Basic theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} # if json_object is supported by the model from praisonaiagents import Agent agent = Agent( instructions="You are a helpful assistant", llm="gemini/gemini-1.5-flash-8b", reflection=True, ) agent.start("Why sky is Blue?") ``` ```python Advanced theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} # if json_object is not supported by the model from praisonaiagents import Agent # Detailed LLM configuration llm_config = { "model": "gemini/gemini-1.5-flash-latest", # Model name without provider prefix # Core settings "temperature": 0.7, # Controls randomness (like temperature) "timeout": 30, # Timeout in seconds "top_p": 0.9, # Nucleus sampling parameter "max_tokens": 1000, # Max tokens in response # Advanced parameters "presence_penalty": 0.1, # Penalize repetition of topics (-2.0 to 2.0) "frequency_penalty": 0.1, # Penalize token repetition (-2.0 to 2.0) # API settings (optional) "api_key": None, # Your API key (or use environment variable) "base_url": None, # Custom API endpoint if needed # Response formatting "response_format": { # Force specific response format "type": "text" # Options: "text", "json_object" }, # Additional controls "seed": 42, # For reproducible responses "stop_phrases": ["##", "END"], # Custom stop sequences } agent = Agent( instructions="You are a helpful Assistant specialized in scientific explanations. " "Provide clear, accurate, and engaging responses.", llm=llm_config, # Pass the detailed configuration # Enable detailed output # Format responses in markdown reflection=True, # Enable self-reflection max_iterations=3, # Maximum reflection iterations min_iterations=1 # Minimum reflection iterations ) # Test the agent response = agent.start("Why is the sky blue? Please explain in simple terms.") ``` ```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} export OPENAI_BASE_URL=http://localhost:11434/v1 ``` ```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} export OPENAI_API_KEY="${GROQ_API_KEY:?Set GROQ_API_KEY in your shell}" export OPENAI_BASE_URL=https://api.groq.com/openai/v1 ``` ```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} export OPENAI_API_KEY="${GEMINI_API_KEY:?Set GEMINI_API_KEY in your shell}" export OPENAI_BASE_URL=https://generativelanguage.googleapis.com/v1beta/openai/ ``` ```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} export OPENAI_BASE_URL=http://localhost:1337/v1 ``` ```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} export OPENAI_BASE_URL=http://localhost:1234/v1 ``` ```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} export OPENAI_API_KEY="${OPENROUTER_API_KEY:?Set OPENROUTER_API_KEY in your shell}" export OPENAI_BASE_URL=https://openrouter.ai/api/v1 ``` ## Provider Auto-Detection (no-config first run) When you run `praisonai run` without setting `--model` or a `model:` key in `config.yaml`, PraisonAI inspects which supported provider credential is present in your environment and picks a provider-appropriate default — so a user whose only key is `ANTHROPIC_API_KEY` no longer gets an OpenAI auth error on first run. Detection is **catalogue-driven**: **any** provider in `PROVIDER_ENV_CATALOGUE` (`praisonaiagents/llm/catalogue.py`) with its API-key env var set is recognised — not just the historical handful. The ordered preference is preserved (OpenAI still wins when multiple keys are set), and the terminal fallback is still `gpt-4o-mini`. | Credential env var(s) | Resolved default model | | -------------------------------------------- | ---------------------------------------------------------------- | | `OPENAI_API_KEY` | `gpt-4o-mini` | | `ANTHROPIC_API_KEY` | `anthropic/claude-3-5-sonnet-latest` | | `GEMINI_API_KEY` | `gemini/gemini-1.5-flash` | | `GOOGLE_API_KEY` | `google/gemini-1.5-flash` | | `GROQ_API_KEY` | `groq/llama-3.3-70b-versatile` | | `COHERE_API_KEY` | `cohere/command-r` | | `OPENROUTER_API_KEY` | `openrouter/openai/gpt-4o-mini` | | `OLLAMA_HOST` | `ollama/llama3.2` | | `MISTRAL_API_KEY` | `mistral/mistral-large-latest` | | `DEEPSEEK_API_KEY` | `deepseek/deepseek-chat` | | `XAI_API_KEY` | `xai/grok-2-latest` | | `TOGETHER_API_KEY` / `TOGETHERAI_API_KEY` | `together_ai/meta-llama/Llama-3.3-70B-Instruct-Turbo` | | `PERPLEXITYAI_API_KEY` | `perplexity/sonar` | | `FIREWORKS_API_KEY` / `FIREWORKS_AI_API_KEY` | `fireworks_ai/accounts/fireworks/models/llama-v3p3-70b-instruct` | | (none of the above) | `gpt-4o-mini` | Precedence: the first catalogued credential that is set wins, in the order above (the head of the catalogue preserves the historical preference). If multiple provider keys are set, the one listed first takes effect. Setting `llm="local"` on an `Agent` bypasses this credential catalogue entirely and runs the [Local Model Resolver](/docs/features/local-model-resolver) — it discovers a running local server, picks the best model, and configures itself with no key. That is the **explicit**, user-driven local path; the keyless fallback below is the **implicit** one. The same resolver drives implicit defaults for `praisonai run`, `praisonai chat`, `praisonai code`, `praisonai init` scaffolding, `praisonai setup`, and the bare-`praisonai` TUI launch — not just `run`. `praisonai code` now flows through the same resolver and first-run gate. ```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} graph TB Start([praisonai run — no model set]) --> A{OPENAI_API_KEY?} A -->|Yes| M1[gpt-4o-mini] A -->|No| B{ANTHROPIC_API_KEY?} B -->|Yes| M2[anthropic/claude-3-5-sonnet-latest] B -->|No| C{GEMINI_API_KEY?} C -->|Yes| M3[gemini/gemini-1.5-flash] C -->|No| D{GOOGLE_API_KEY?} D -->|Yes| M4[google/gemini-1.5-flash] D -->|No| E{GROQ_API_KEY?} E -->|Yes| M5[groq/llama-3.3-70b-versatile] E -->|No| F{COHERE_API_KEY?} F -->|Yes| M6[cohere/command-r] F -->|No| G{OLLAMA_HOST?} G -->|Yes| M7[ollama/llama3.2] G -->|No| L{Local endpoint reachable?} L -->|Yes| M9[ollama/detected] L -->|No| M8[gpt-4o-mini fallback] classDef start fill:#6366F1,stroke:#7C90A0,color:#fff classDef check fill:#F59E0B,stroke:#7C90A0,color:#fff classDef model fill:#10B981,stroke:#7C90A0,color:#fff classDef fallback fill:#8B0000,stroke:#7C90A0,color:#fff class Start start class A,B,C,D,E,F,G,L check class M1,M2,M3,M4,M5,M6,M7,M9 model class M8 fallback ``` ### Same resolver from the Python SDK Constructing `Agent(instructions="…")` **without** `llm=` uses the same `_PROVIDER_DEFAULT_MODELS` table — with `OLLAMA_HOST` set alone, the Agent picks `ollama/llama3.2` and routes it through litellm. ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} import os from praisonaiagents import Agent os.environ["OLLAMA_HOST"] = "http://localhost:11434" # No llm= argument, no OPENAI_API_KEY — the Agent picks ollama/llama3.2 # and routes through litellm. agent = Agent(instructions="Answer briefly.") agent.start("2+2") ``` Before [PR #4795](https://github.com/MervinPraison/PraisonAI/pull/4795), six of the seven defaults resolved the right model name but were then routed to the native OpenAI client, causing `ValueError: OPENAI_API_KEY environment variable is required`. This is fixed as of #4795 — the SDK `Agent()` constructor now matches `praisonai run`. See [Agent Model Resolution](/docs/features/agent-model-resolution) for the full flow. ### Keyless local-first fallback (no env vars set) If no cloud provider key is set **and** a local OpenAI-compatible endpoint answers on `http://127.0.0.1:11434` (or wherever `OPENAI_BASE_URL` / `OLLAMA_HOST` points), PraisonAI uses that local server as the zero-config default — the first `praisonai run "..."` just works before you configure anything. ```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} # Terminal 1 ollama serve ollama pull llama3.2 # Terminal 2 — no OPENAI_API_KEY, no OLLAMA_HOST, nothing set praisonai run "hello" # > No cloud key found; using local model ollama/llama3.2. # > Run `praisonai setup` to add a hosted provider. ``` Precedence (first match wins): 1. `--model ` on the command line. 2. Any cloud provider key from the table above. 3. A reachable local endpoint at `OPENAI_BASE_URL` → `OLLAMA_HOST` → `http://127.0.0.1:11434`. 4. `gpt-4o-mini` fallback. Detection is timeout-bounded (\~150 ms) and cached briefly, so the first-run hot path stays fast when nothing is listening. Cloud keys always win — the local probe is skipped entirely when any cloud key is set. Detection recognises two shapes at the local server: | Server type | Probed path | Resolved model id | | ------------------------------------------------------ | ------------------------------------------------- | ------------------------------------------------------- | | Ollama | `GET /api/tags` (at the root, not `/v1/api/tags`) | `ollama/` | | Generic OpenAI-compatible (llama.cpp, LM Studio, vLLM) | `GET /v1/models` | `openai/` (routed through the local base URL) | See [Local Models](/docs/features/local-models) for the full guide. This keyless fallback is the **implicit** local path. To request a local model explicitly — with full engine discovery and a loud failure when a named server is missing — set `llm="local"` on the `Agent`. See the [Local Model Resolver](/docs/features/local-model-resolver). An explicit `--model ` flag or a `model:` key in `config.yaml` always overrides auto-detection. A local `base_url` no longer captures hosted model names — `gpt-*`, `claude*`, and `gemini-*` keep their native adapter even behind `http://localhost:11434/v1`. See [Mixing local and hosted models](/docs/features/local-models#mixing-local-and-hosted-models). No cloud key at all? If Ollama (or any OpenAI-compatible local server) is running, PraisonAI will use it — no API key and no config. See [Keyless Local-First Run](/docs/features/keyless-local-first-run). ## Routed model provider detection When a model id routes through Bedrock, Vertex AI, or OpenRouter to reach Claude or Gemini, PraisonAI detects the true provider from the model id — not the routing prefix — so the correct streaming adapter is used. | Routed model id | Detected provider | | ------------------------------------------ | ----------------- | | `bedrock/anthropic.claude-3-5-sonnet-v1:0` | Anthropic | | `vertex_ai/claude-3-5-sonnet@20240620` | Anthropic | | `openrouter/anthropic/claude-3.5-sonnet` | Anthropic | | `vertex_ai/gemini-1.5-pro` | Gemini | | `openrouter/google/gemini-1.5-pro` | Gemini | | `gpt-4o`, `openai/...` | OpenAI | Async streaming (`agent.astart(...)`) and `agent.astream(...)` now work correctly for these routed model ids. Older versions fell through to the OpenAI streaming adapter and raised a runtime error on the async path. Detection first checks known provider prefixes, then falls back to substring matching: `"claude"` or `"anthropic"` in the model id resolves to Anthropic, and `"gemini"` resolves to Gemini. Genuinely-OpenAI ids (`gpt-4o`, `openai/...`) are unaffected. ## Primary vs Auxiliary Model PraisonAI splits your model choice into two knobs: | Knob | What it drives | Configured via | | ------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------- | | `model` | The agent's own reasoning / tool-use turns | `Agent(llm=…)`, `[defaults].model`, `MODEL_NAME` / `OPENAI_MODEL_NAME` env | | `small_model` / auxiliary | Cheap internal calls — session titles, context compaction/summarisation, LLM guardrail validation, memory quality scoring, workflow routing, `lite` helpers | `[defaults].small_model`, or the `PRAISONAI_AUXILIARY_MODEL` env var | The auxiliary knob falls back to `model` when unset, so single-provider (Anthropic, Ollama, on-prem) setups make **zero** unexpected third-party calls — the auxiliary calls all stay on your configured provider. Two config surfaces resolve the auxiliary model: * **Config-file resolver** (`[defaults].small_model`) — used by session-title generation via `get_small_model(primary_model, fallback)`. * **Env-var resolver** (`PRAISONAI_AUXILIARY_MODEL` → `OPENAI_MODEL_NAME` → `gpt-4o-mini`) — used by memory quality scoring, context compaction, workflow routing, `lite` helpers, task callbacks, and learn-manager extraction. Session-title generation *also* falls through to this ladder when `defaults.small_model` is unset. The env-var resolver covers **8 modules**: `memory/memory.py`, `memory/learn/manager.py`, `context/compressor.py`, `context/optimizer.py`, `workflows/workflows.py`, `lite/__init__.py`, `task/task.py`, and `session/title.py`. Cost tables (`utils/cost_utils.py`, `llm/_cost.py`), docstring examples, and public dataclass field defaults are **deliberately not routed** — `PRAISONAI_AUXILIARY_MODEL` does not change pricing behaviour. Empty or whitespace-only env values (`OPENAI_MODEL_NAME=" "`) are ignored. ```toml theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} # .praisonai/config.toml [defaults] model = "gpt-4o" small_model = "gpt-4o-mini" ``` ```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} # Or purely via environment — no config file required export OPENAI_MODEL_NAME=ollama/llama3.3:70b # primary agent model export PRAISONAI_AUXILIARY_MODEL=ollama/qwen3:0.6b # cheap helper model ``` See [Configuration File → Cheap auxiliary model for internal calls](/docs/docs/features/config-file#cheap-auxiliary-model-for-internal-calls) for the full precedence ladder. ## Supported Models for No Code | PraisonAI Chat | PraisonAI Code | PraisonAI (Multi-Agents) | | ---------------------------------------------------- | ---------------------------------------------------- | ------------------------ | | [Litellm](https://litellm.vercel.app/docs/providers) | [Litellm](https://litellm.vercel.app/docs/providers) | Below Models | * [OpenAI](models/openai.md) * [Groq](models/groq.md) * [Google Gemini](models/google.md) * [Anthropic Claude](models/anthropic.md) * [Cohere](models/cohere.md) * [Mistral](models/mistral.md) * [Ollama](models/ollama.md) * [Other Models](models/other.md) ## Example agents.yaml This uses Multi-Agents with Multi-LLMs. ```yaml theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} framework: crewai topic: research about the causes of lung disease agents: # Canonical: use 'agents' instead of 'roles' research_analyst: instructions: # Canonical: use 'instructions' instead of 'backstory' Experienced in analyzing scientific data related to respiratory health. goal: Analyze data on lung diseases role: Research Analyst llm: model: "groq/llama3-70b-8192" function_calling_llm: model: "google/gemini-1.5-flash-001" tasks: data_analysis: description: Gather and analyze data on the causes and risk factors of lung diseases. expected_output: Report detailing key findings on lung disease causes. tools: - 'InternetSearchTool' medical_writer: instructions: # Canonical: use 'instructions' instead of 'backstory' Skilled in translating complex medical information into accessible content. goal: Compile comprehensive content on lung disease causes role: Medical Writer llm: model: "anthropic/claude-3-haiku-20240307" function_calling_llm: model: "openai/gpt-4o" tasks: content_creation: description: Create detailed content summarizing the research findings on lung disease causes. expected_output: Document outlining various causes and risk factors of lung diseases. tools: - '' editor: instructions: # Canonical: use 'instructions' instead of 'backstory' Proficient in editing medical content for accuracy and clarity. goal: Review and refine content on lung disease causes role: Editor llm: model: "cohere/command-r" tasks: content_review: description: Edit and refine the compiled content on lung disease causes for accuracy and coherence. expected_output: Finalized document on lung disease causes ready for dissemination. tools: - '' dependencies: [] ``` ## How It Works The Agent passes your `llm` value to the provider resolver, which routes the request to the matching model and returns the response. ```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}} sequenceDiagram participant User participant Agent participant Model as Provider User->>Agent: Agent(llm="gpt-4o-mini").start("…") Agent->>Model: Route request to the resolved provider Model-->>Agent: Completion Agent-->>User: Final answer ``` ## Best Practices Skip `llm=` on first runs. PraisonAI resolves a sensible default from whichever provider key is set — see [Provider Auto-Detection](#provider-auto-detection-no-config-first-run). Pass `llm="gemini/gemini-1.5-flash-8b"` or `llm="anthropic/claude-3-5-sonnet-latest"` to target a specific provider model. Set provider keys in your shell or `.env`. Use `api_key=None` in `llm_config` so the SDK reads the environment variable. Use a small fast model (`gpt-4o-mini`, `gemini-1.5-flash`) for routing and a larger model only where quality matters. ## Related Run your first agent in a few lines. Give models real actions with tools. Point an Agent at any running local server with `llm="local"`. The implicit local fallback when no cloud key is set.