LLM Config

Configure primary and fallback LLM models with provider prefixes and API settings

LLM Config lets you set the model, API endpoint, and fallback chain so your agent always has a working model.

from praisonaiagents import Agent
from praisonaiagents.config import LLMConfig

agent = Agent(
    name="Assistant",
    instructions="You are a helpful assistant.",
    llm_config=LLMConfig(
        model="gpt-4o",
        fallback_models=["claude-3-5-sonnet-20241022", "gpt-4o-mini"],
    ),
)

result = agent.start("Explain quantum computing simply")
print(result)

The user sends a prompt; the agent tries the primary model and transparently falls back if the provider fails.

Quick Start

1. Primary Model Only

   Set just the model to get started:

   from praisonaiagents import Agent
   from praisonaiagents.config import LLMConfig
   
   agent = Agent(
       name="Assistant",
       instructions="You are a helpful assistant.",
       llm_config=LLMConfig(model="gpt-4o")
   )
   
   agent.start("What is the speed of light?")
   
2. With Fallback Chain

   Add fallback models for reliability — if the primary fails, the agent automatically tries the next:

   from praisonaiagents import Agent
   from praisonaiagents.config import LLMConfig
   
   agent = Agent(
       name="ResilientAssistant",
       instructions="You are a helpful assistant.",
       llm_config=LLMConfig(
           model="gpt-4o",
           fallback_models=["claude-3-5-sonnet-20241022", "gpt-4o-mini"],
       ),
   )
   agent.start("Summarize today's top AI research papers.")
   
3. With LiteLLM Provider Prefix

   from praisonaiagents import Agent
   from praisonaiagents.config import LLMConfig
   
   agent = Agent(
       instructions="You are a helpful assistant.",
       llm_config=LLMConfig(
           model="anthropic/claude-3-5-sonnet-20241022",
           fallback_models=["openai/gpt-4o", "openai/gpt-4o-mini"],
       ),
   )
   agent.start("Write a technical blog post about microservices.")
   

---

How It Works

Phase	What happens
1. Primary	Agent calls the configured model first
2. Fallback	On failure, tries each model in fallback_models in order
3. Response	First successful response is returned

---

OPENAI_MODEL_NAME with a provider prefix

A provider/model value in OPENAI_MODEL_NAME now routes through litellm, matching Agent(llm="provider/model").

import os
from praisonaiagents import Agent

os.environ["OPENAI_MODEL_NAME"] = "ollama/llama3.2"

# Routes through litellm — same as Agent(llm="ollama/llama3.2").
agent = Agent(instructions="Answer briefly.")
agent.start("2+2")

As of PR #4795, OPENAI_MODEL_NAME="ollama/x" (or any provider/model value) routes to litellm, matching Agent(llm="ollama/x"). Before the fix, those two paths disagreed — an OPENAI_MODEL_NAME with a provider prefix ended up on the native OpenAI client with a model name OpenAI never heard of. A bare value like OPENAI_MODEL_NAME="gpt-4o" still uses the native OpenAI client.

---

Configuration Options

LLMConfig SDK Reference

Full parameter reference for LLMConfig

Option	Type	Default	Description
model	str	(required)	Primary model to use
fallback_models	list[str] | None	None	Ordered fallback models
base_url	str | None	None	API endpoint override
api_key	str | None	None	API key (defaults to env vars)
auth	dict | None	None	Additional auth headers

---

Common Patterns

Pattern 1 — Self-hosted model via custom endpoint

from praisonaiagents import Agent
from praisonaiagents.config import LLMConfig

agent = Agent(
    instructions="You are a helpful assistant.",
    llm_config=LLMConfig(
        model="llama3.2",
        base_url="http://localhost:11434/v1",
        api_key="ollama",
    ),
)
response = agent.start("What is the capital of France?")
print(response)

Pattern 2 — Multi-provider fallback for resilience

from praisonaiagents import Agent
from praisonaiagents.config import LLMConfig

agent = Agent(
    name="PrivateModelAgent",
    instructions="You are a helpful assistant.",
    llm_config=LLMConfig(
        model="my-custom-model",
        base_url="https://my-llm-gateway.example.com/v1",
        api_key="sk-my-private-key",
        auth={"X-Organization-ID": "org-123"},
    )
)

Self-hosted model via Ollama:

from praisonaiagents import Agent
from praisonaiagents.config import LLMConfig

agent = Agent(
    name="LocalAgent",
    instructions="You are a helpful assistant.",
    llm_config=LLMConfig(
        model="ollama/llama3.2",
        base_url="http://localhost:11434",
    )
)

Pattern 3 — Multi-provider fallback for business continuity

from praisonaiagents import Agent
from praisonaiagents.config import LLMConfig

agent = Agent(
    name="ResilientAgent",
    instructions="You are a reliable assistant for critical tasks.",
    llm_config=LLMConfig(
        model="openai/gpt-4o",
        fallback_models=[
            "anthropic/claude-3-5-sonnet-20241022",
            "openai/gpt-4o-mini",
        ],
    ),
)
agent.start("Generate a business continuity plan for our e-commerce platform.")

Pattern 4 — Passing LLMConfig to a team or flow

from praisonaiagents import Agent, AgentTeam
from praisonaiagents.config import LLMConfig

researcher = Agent(name="Researcher", instructions="Research.")
writer     = Agent(name="Writer",     instructions="Write.")

# The container unwraps LLMConfig to its .model string,
# then fills in members that didn't name a model themselves.
team = AgentTeam(
    agents=[researcher, writer],
    llm=LLMConfig(model="anthropic/claude-3-5-sonnet-20241022"),
)
team.start("Summarise the latest AI safety research")

Containers keep only .model from the LLMConfig.

base_url, api_key, and fallback_models are dropped at the container level — pin those on individual Agent(s) that need them.

---

Best Practices

Always add at least one fallback model

Even one fallback prevents your agent from failing completely during rate limits or outages. Use a smaller, cheaper model as the fallback — it costs less and is often available when the primary isn’t.

Use environment variables for API keys

Never hardcode api_key in your code. Leave it as None and set OPENAI_API_KEY, ANTHROPIC_API_KEY, etc. in your environment instead. LLMConfig picks them up automatically via LiteLLM.

Use LiteLLM prefixes for multi-provider setups

Prefix models with the provider name (anthropic/, openai/, ollama/) to avoid ambiguity when using multiple providers in a fallback chain.

Test your fallback chain

Deliberately trigger a failure (e.g., invalid primary model name) and verify your fallback fires correctly before going to production.

For team-wide endpoint or fallback config, pin per-agent

AgentTeam(llm=LLMConfig(...)) and AgentFlow(llm=LLMConfig(...)) only propagate the .model string today. If every member should share a custom base_url / api_key / fallback_models, set an LLMConfig on each member Agent rather than on the container.

---

Related

LLM Gateways

Route requests through a unified LLM gateway

LLM Endpoint Config

Fine-tune endpoint parameters per model

LLM Error Classification

How errors trigger fallback in the chain

Failover

Automatic failover strategies for reliability