Other Models

Use any LiteLLM-supported model with PraisonAI Agents

Overview

PraisonAI uses LiteLLM under the hood, supporting 100+ LLM providers. Use the format provider/model-name for any supported model.

LiteLLM Provider Format

Provider	Format	Example
OpenAI	gpt-* or openai/*	gpt-4o, openai/gpt-4o
Anthropic	claude-*	claude-sonnet-4-5
Google	gemini/*	gemini/gemini-2.5-flash
Azure	azure/*	azure/gpt-4
AWS Bedrock	bedrock/*	bedrock/anthropic.claude-3-5-sonnet
Vertex AI	vertex_ai/*	vertex_ai/gemini-pro
Hugging Face	huggingface/*	huggingface/meta-llama/Llama-2-7b
Together AI	together_ai/*	together_ai/togethercomputer/llama-2-70b
Replicate	replicate/*	replicate/meta/llama-2-70b
Anyscale	anyscale/*	anyscale/meta-llama/Llama-2-70b

Routed Claude / Gemini detection

When Bedrock, Vertex AI, or OpenRouter routes to Claude or Gemini, PraisonAI detects the true provider from the model id, so the correct adapter is used and async streaming works.

Routed model id	Detected provider	Async streaming
bedrock/anthropic.claude-3-5-sonnet-v1:0	Anthropic	✅ supported
vertex_ai/claude-3-5-sonnet@20240620	Anthropic	✅ supported
vertex_ai/gemini-1.5-pro	Gemini	✅ supported
openrouter/anthropic/claude-3.5-sonnet	Anthropic	✅ supported
openrouter/google/gemini-1.5-pro	Gemini	✅ supported

See Routed model provider detection for the full rules.

Python (Generic Pattern)

# Set the appropriate API key for your provider
# export PROVIDER_API_KEY=your-api-key
from praisonaiagents import Agent

agent = Agent(
    instructions="You are a helpful assistant",
    llm="provider/model-name"  # Replace with your provider/model
)
agent.start("Hello, how can you help me?")

OpenAI-Compatible Endpoints

# Simplest: pass base_url and api_key at the top level.
# Works with any OpenAI-compatible /v1 host — a bare catalog model id is fine.
from praisonaiagents import Agent

agent = Agent(
    name="assistant",
    instructions="You are a helpful assistant",
    llm="deepseek-v4-flash",
    base_url="https://api.pzero.studio/v1",
    api_key="",
)
agent.start("Hello")

See OpenAI-Compatible Endpoints for a full walkthrough.

# Equivalent dict form
from praisonaiagents import Agent

agent = Agent(
    instructions="You are a helpful assistant",
    llm={
        "model": "your-model-name",
        "api_base": "https://your-api-endpoint.com/v1",
        "api_key": "your-api-key"
    }
)
agent.start("What can you do?")

LM Studio (Local)

# LM Studio runs on localhost:1234 by default
from praisonaiagents import Agent

agent = Agent(
    instructions="You are a helpful assistant",
    llm={
        "model": "local-model",
        "api_base": "http://localhost:1234/v1",
        "api_key": "not-needed"
    }
)
agent.start("Explain AI")

vLLM Server

# vLLM OpenAI-compatible server
from praisonaiagents import Agent

agent = Agent(
    instructions="You are a helpful assistant",
    llm={
        "model": "meta-llama/Llama-2-7b-hf",
        "api_base": "http://localhost:8000/v1",
        "api_key": "not-needed"
    }
)
agent.start("What is machine learning?")

CLI

# Generic pattern
python -m praisonai "Your prompt" --llm provider/model-name

# With custom endpoint
export OPENAI_API_BASE=http://localhost:1234/v1
export OPENAI_API_KEY=not-needed
python -m praisonai "Your prompt" --llm local-model

# Run agents.yaml
python -m praisonai

YAML

framework: praisonai
topic: Custom model usage
agents:
  assistant:
    role: General Assistant
    goal: Help with various tasks
    instructions: You are a helpful assistant
    llm:
      model: provider/model-name  # Replace with your provider/model
    tasks:
      help_task:
        description: Assist with the user's request
        expected_output: Helpful response

Custom Endpoint YAML

framework: praisonai
topic: Local model usage
agents:
  assistant:
    role: Local Assistant
    goal: Help with tasks using local model
    instructions: You are a helpful assistant
    llm:
      model: local-model
      api_base: http://localhost:1234/v1
      api_key: not-needed
    tasks:
      help_task:
        description: Assist with the user's request
        expected_output: Helpful response

Environment Variables

Common environment variables for different providers:

# OpenAI
export OPENAI_API_KEY=your-key

# Anthropic
export ANTHROPIC_API_KEY=your-key

# Google
export GEMINI_API_KEY=your-key

# Azure
export AZURE_API_KEY=your-key
export AZURE_API_BASE=https://your-resource.openai.azure.com

# AWS Bedrock
export AWS_ACCESS_KEY_ID=your-key
export AWS_SECRET_ACCESS_KEY=your-secret
export AWS_REGION=us-east-1

# Custom OpenAI-compatible
export OPENAI_API_BASE=http://your-endpoint/v1
export OPENAI_API_KEY=your-key

Resources

• LiteLLM Providers - Full list of supported providers
• LiteLLM Models - Model discovery and pricing