LLM Endpoint Configuration

Point your agents at any OpenAI-compatible endpoint with environment variables

Configure where your agents send LLM requests using environment variables, with automatic provider-specific routing for major LLM providers.

from praisonaiagents import Agent

agent = Agent(name="assistant", llm="anthropic/claude-sonnet-4-6")
agent.start("Draft a release note for today's deploy.")

The user sets MODEL_NAME and provider keys; PraisonAI resolves the OpenAI-compatible endpoint before the agent runs.

As of PraisonAI 4.6.106, the endpoint resolver is available directly from praisonai-code:

from praisonai_code.llm.env import LLMEndpoint, resolve_llm_endpoint

The legacy path from praisonai.llm.env import ... still works via a sys.modules shim — both resolve to the same object.

Quick Start

1. Use OpenAI (default)

   Set your OpenAI API key and create an agent:

   import os
   from praisonaiagents import Agent
   
   if not os.getenv("OPENAI_API_KEY"):
       raise EnvironmentError("Set OPENAI_API_KEY in your environment")
   
   agent = Agent(
       name="Research Assistant",
       instructions="You are a helpful research assistant"
   )
   
   result = agent.start("Explain quantum computing in simple terms")
   
2. Use Anthropic directly

   No base URL needed - automatic provider routing:

   import os
   from praisonaiagents import Agent
   
   os.environ["MODEL_NAME"] = "anthropic/claude-3-5-sonnet"
   if not os.getenv("ANTHROPIC_API_KEY"):
       raise EnvironmentError("Set ANTHROPIC_API_KEY in your environment")
   
   agent = Agent(
       name="Claude Assistant",
       instructions="You are Claude, an AI assistant"
   )
   
   result = agent.start("What makes you different from other AI models?")
   
3. Use Groq for fast inference

   High-speed inference with automatic routing:

   import os
   from praisonaiagents import Agent
   
   os.environ["MODEL_NAME"] = "groq/llama3-70b"
   if not os.getenv("GROQ_API_KEY"):
       raise EnvironmentError("Set GROQ_API_KEY in your environment")
   
   agent = Agent(
       name="Fast Assistant",
       instructions="You are a speed-optimized assistant"
   )
   
   result = agent.start("Generate a quick summary of machine learning")
   
4. Use Google Gemini

   Access Google’s latest models directly:

   import os
   from praisonaiagents import Agent
   
   os.environ["MODEL_NAME"] = "google/gemini-1.5-pro"
   if not os.getenv("GOOGLE_API_KEY"):
       raise EnvironmentError("Set GOOGLE_API_KEY in your environment")
   
   agent = Agent(
       name="Gemini Assistant",
       instructions="You are powered by Google Gemini"
   )
   
   result = agent.start("Analyze this complex dataset")
   

---

How It Works

---

Provider-specific defaults

If you set MODEL_NAME=anthropic/claude-3-5-sonnet, you do not need to set OPENAI_BASE_URL — the right base URL is picked automatically.

Model prefix	API key env var	Default base URL
anthropic/	ANTHROPIC_API_KEY	https://api.anthropic.com/v1
google/	GOOGLE_API_KEY	https://generativelanguage.googleapis.com/v1beta
gemini/	GEMINI_API_KEY	https://generativelanguage.googleapis.com/v1beta
groq/	GROQ_API_KEY	https://api.groq.com/openai/v1
cohere/	COHERE_API_KEY	https://api.cohere.ai/v1
openrouter/	OPENROUTER_API_KEY	https://openrouter.ai/api/v1
ollama/	OLLAMA_API_KEY	http://localhost:11434/v1
no prefix match	OPENAI_API_KEY	https://api.openai.com/v1

Provider keys do not cross-fallback. If you use anthropic/claude-3-5-sonnet and only OPENAI_API_KEY is set, the call has no credentials. This is a security fix, not a bug — it prevents accidental credential exposure.

---

Environment Variables

Variable	Purpose	Precedence
MODEL_NAME	Model name (highest priority)	1
OPENAI_MODEL_NAME	Model name (legacy compat)	2
OPENAI_BASE_URL	LLM endpoint URL (highest priority)	1
OPENAI_API_BASE	LLM endpoint URL (legacy compat)	2
OLLAMA_API_BASE	Ollama endpoint URL	3
ANTHROPIC_API_KEY	Anthropic API key (for anthropic/* models)	—
GOOGLE_API_KEY	Google API key (for google/* models)	—
GEMINI_API_KEY	Gemini API key (for gemini/* models)	—
GROQ_API_KEY	Groq API key (for groq/* models)	—
COHERE_API_KEY	Cohere API key (for cohere/* models)	—
OPENROUTER_API_KEY	OpenRouter API key (for openrouter/* models)	—
OLLAMA_API_KEY	Ollama API key (for ollama/* models)	—
OPENAI_API_KEY	OpenAI API key (for OpenAI models and fallback)	—

Defaults

Setting	Default Value
Model	gpt-4o-mini
Base URL	Provider-specific or https://api.openai.com/v1
API Key	None

---

Common Patterns

Run against Ollama

bash

export OPENAI_BASE_URL="http://localhost:11434/v1"
export MODEL_NAME="llama3"
python your_agent.py

python

import os
from praisonaiagents import Agent

# Configure Ollama
os.environ["OPENAI_BASE_URL"] = "http://localhost:11434/v1"
os.environ["MODEL_NAME"] = "llama3"

agent = Agent(
    name="Ollama Assistant",
    instructions="You are running on Ollama"
)

result = agent.start("Explain the benefits of local AI")

Run against a corporate OpenAI proxy

bash

export OPENAI_BASE_URL="https://corporate-proxy.company.com/v1"
export OPENAI_API_KEY="${OPENAI_API_KEY:?Set OPENAI_API_KEY in your shell}"
export MODEL_NAME="gpt-4"
python your_agent.py

python

import os
from praisonaiagents import Agent

# Configure corporate proxy
os.environ["OPENAI_BASE_URL"] = "https://corporate-proxy.company.com/v1"
if not os.getenv("OPENAI_API_KEY"):
    raise EnvironmentError("Set OPENAI_API_KEY in your environment")
os.environ["MODEL_NAME"] = "gpt-4"

agent = Agent(
    name="Corporate Assistant",
    instructions="You are using a corporate OpenAI proxy"
)

result = agent.start("Generate a business report")

Use Anthropic directly

bash

export MODEL_NAME="anthropic/claude-3-5-sonnet"
export ANTHROPIC_API_KEY="${ANTHROPIC_API_KEY:?Set ANTHROPIC_API_KEY in your shell}"
python your_agent.py

python

import os
from praisonaiagents import Agent

os.environ["MODEL_NAME"] = "anthropic/claude-3-5-sonnet"
if not os.getenv("ANTHROPIC_API_KEY"):
    raise EnvironmentError("Set ANTHROPIC_API_KEY in your environment")

agent = Agent(
    name="Anthropic Assistant",
    instructions="You are powered by Claude"
)

result = agent.start("Write a thoughtful analysis")

Use Groq for speed

bash

export MODEL_NAME="groq/llama3-70b"
export GROQ_API_KEY="${GROQ_API_KEY:?Set GROQ_API_KEY in your shell}"
python your_agent.py

python

import os
from praisonaiagents import Agent

os.environ["MODEL_NAME"] = "groq/llama3-70b"
if not os.getenv("GROQ_API_KEY"):
    raise EnvironmentError("Set GROQ_API_KEY in your environment")

agent = Agent(
    name="Speed Assistant",
    instructions="You provide fast responses"
)

result = agent.start("Quick summary of quantum physics")

Use Google Gemini

bash

export MODEL_NAME="google/gemini-1.5-pro"
export GOOGLE_API_KEY="${GOOGLE_API_KEY:?Set GOOGLE_API_KEY in your shell}"
python your_agent.py

python

import os
from praisonaiagents import Agent

os.environ["MODEL_NAME"] = "google/gemini-1.5-pro"
if not os.getenv("GOOGLE_API_KEY"):
    raise EnvironmentError("Set GOOGLE_API_KEY in your environment")

agent = Agent(
    name="Gemini Assistant",
    instructions="You are powered by Google Gemini"
)

result = agent.start("Analyze this complex problem")

Use Azure OpenAI / Bedrock via LiteLLM

bash

export OPENAI_BASE_URL="https://your-litellm-proxy/v1"
export OPENAI_API_KEY="${OPENAI_API_KEY:?Set OPENAI_API_KEY in your shell}"
export MODEL_NAME="azure/gpt-4"
python your_agent.py

python

import os
from praisonaiagents import Agent

# Configure LiteLLM for Azure
os.environ["OPENAI_BASE_URL"] = "https://your-litellm-proxy/v1"
if not os.getenv("OPENAI_API_KEY"):
    raise EnvironmentError("Set OPENAI_API_KEY in your environment")
os.environ["MODEL_NAME"] = "azure/gpt-4"

agent = Agent(
    name="Azure Assistant",
    instructions="You are using Azure OpenAI via LiteLLM"
)

result = agent.start("Analyze this data")

---

User Interaction Flow

---

Best Practices

Set OPENAI_BASE_URL, not OPENAI_API_BASE

OPENAI_BASE_URL is the standard OpenAI SDK environment variable and has the highest precedence. Use this for all new configurations rather than the legacy OPENAI_API_BASE.

Empty string ≠ unset

An empty string value is skipped during resolution, and the next variable in precedence order is tried. To disable a variable, unset it completely rather than setting it to an empty string.

# This skips OPENAI_BASE_URL and tries OPENAI_API_BASE
export OPENAI_BASE_URL=""
export OPENAI_API_BASE="https://proxy.com/v1"

# This uses OPENAI_BASE_URL
unset OPENAI_BASE_URL
export OPENAI_API_BASE="https://proxy.com/v1"

Use .env files for local dev

Create a .env file in your project root for local development:

# .env
OPENAI_BASE_URL=http://localhost:11434/v1
MODEL_NAME=llama3
# OPENAI_API_KEY not needed for Ollama

Load it in your Python code:

from dotenv import load_dotenv
load_dotenv()

from praisonaiagents import Agent
# Environment variables are now loaded

Realtime/WebSocket endpoints

For realtime features, WebSocket URLs are auto-derived from HTTP URLs. The system automatically:

• Converts https:// to wss://
• Strips /v1 suffix to avoid /v1/v1/realtime
• Appends the appropriate realtime path

You only need to set OPENAI_BASE_URL - the realtime endpoint is handled automatically.

---

Related

AutoGen Config List

Build an AutoGen-style config_list from the resolved endpoint

Standalone LLM Modules

All LLM modules available without the wrapper package

LLM Configuration

Complete LLM configuration options

Models

Supported models and providers

Provider Pages

Anthropic

Claude models and configuration

Groq

High-speed inference setup

Google

Gemini models and API access

Cohere

Language model configuration

OpenRouter

Multi-provider router setup

Ollama

Local model deployment

C7 note: As of PR #2550, the tool-discovery pipeline (tool_resolver), safe loader (_safe_loader), framework probes (_framework_availability), and plugin registry (tool_registry) also moved to praisonai_code — completing the same C7 arc as LLM config. See Tool Discovery Order and Local Tools Loading.