OpenAI-Compatible Endpoints

Point PraisonAI at any /v1 host with base_url + api_key — no provider prefix needed

Point PraisonAI at any OpenAI-compatible /v1 host by setting base_url and api_key on the Agent — bare catalog model ids are fine, no provider/ prefix required.

Running the server locally? If you don’t want to hard-code the base URL or model name, use llm="local" — it discovers a running local server (Ollama, llama.cpp, LM Studio, vLLM, …) and configures itself. See the Local Model Resolver.

Quick Start

1. Simplest form

   Pass base_url and api_key at the top level with a bare model id.

   from praisonaiagents import Agent
   
   agent = Agent(
       name="assistant",
       instructions="You are a helpful assistant",
       llm="deepseek-v4-flash",
       base_url="https://api.pzero.studio/v1",
       api_key="your-key-or-empty-if-public",
   )
   agent.start("Explain what an OpenAI-compatible endpoint is in one paragraph.")
   
2. Environment-variable variant

   Set the host once with environment variables, then keep the code clean.

   export OPENAI_API_BASE=https://api.pzero.studio/v1
   export OPENAI_API_KEY=your-key
   

   from praisonaiagents import Agent
   
   agent = Agent(
       name="assistant",
       instructions="You are a helpful assistant",
       llm="deepseek-v4-flash",
   )
   agent.start("Hi")
   
3. Dict form (equivalent)

   The dict form does the same thing when you prefer to bundle connection settings inside llm.

   from praisonaiagents import Agent
   
   agent = Agent(
       name="assistant",
       instructions="You are a helpful assistant",
       llm={
           "model": "deepseek-v4-flash",
           "api_base": "https://api.pzero.studio/v1",
           "api_key": "",
       },
   )
   agent.start("Hello")
   

---

How It Works

PraisonAI routes a bare model id plus a base_url through the OpenAI-compatible client.

Situation	What happens
Bare model + base_url set	Routed as openai/<model> to <base_url>
Prefixed model (e.g. anthropic/…, ollama/…, openai/…)	Passed through unchanged
Bare model with no base_url and no OPENAI_API_BASE	Uses OpenAI default (api.openai.com/v1)

---

Configuration Options

Set these top-level Agent parameters for any OpenAI-compatible host.

Option	Type	Default	Description
llm	str | dict	required	Catalog model id (e.g. "deepseek-v4-flash") or a full dict {model, api_base, api_key}
base_url	str	None	OpenAI-compatible /v1 root, e.g. https://api.pzero.studio/v1
api_key	str	None	API key for the host (may be empty for public catalogs)

Agent SDK Reference

Full parameter surface for the Agent class.

---

Common Patterns

Every host uses the same 5-line pattern — only base_url and the catalog id change.

P0 / pzero.studio

from praisonaiagents import Agent

agent = Agent(
    name="assistant",
    llm="deepseek-v4-flash",
    base_url="https://api.pzero.studio/v1",
    api_key="",
)
agent.start("Hello")

DeepInfra

from praisonaiagents import Agent

agent = Agent(
    name="assistant",
    llm="meta-llama/Meta-Llama-3-8B-Instruct",
    base_url="https://api.deepinfra.com/v1/openai",
    api_key="your-deepinfra-key",
)
agent.start("Hello")

Fireworks

from praisonaiagents import Agent

agent = Agent(
    name="assistant",
    llm="accounts/fireworks/models/llama-v3-8b-instruct",
    base_url="https://api.fireworks.ai/inference/v1",
    api_key="your-fireworks-key",
)
agent.start("Hello")

Together AI

from praisonaiagents import Agent

agent = Agent(
    name="assistant",
    llm="meta-llama/Llama-3-8b-chat-hf",
    base_url="https://api.together.xyz/v1",
    api_key="your-together-key",
)
agent.start("Hello")

vLLM

You can also use the vllm/… / hosted_vllm/… prefix directly — see Local prefix route for the adapter with tool-call repair.

from praisonaiagents import Agent

agent = Agent(
    name="assistant",
    llm="meta-llama/Meta-Llama-3-8B-Instruct",
    base_url="http://localhost:8000/v1",
    api_key="not-needed",
)
agent.start("Hello")

LM Studio

You can also use the lm_studio/… prefix directly — see Local prefix route for the adapter with tool-call repair.

from praisonaiagents import Agent

agent = Agent(
    name="assistant",
    llm="local-model",
    base_url="http://localhost:1234/v1",
    api_key="not-needed",
)
agent.start("Hello")

Company gateway

from praisonaiagents import Agent

agent = Agent(
    name="assistant",
    llm="your-catalog-id",
    base_url="https://gateway.company.internal/v1",
    api_key="your-key",
)
agent.start("Hello")

---

Local prefix route (tool-call repair)

For a locally-served OpenAI-compatible server — LM Studio, vLLM, llama.cpp’s llama-server — use a dedicated prefix instead of base_url + bare model. PraisonAI recognises these prefixes and selects a LocalOpenAIAdapter that keeps the standard OpenAI message shape and adds a tool-call repair budget of 2 — a safety net for the malformed JSON tool calls small local models occasionally emit.

from praisonaiagents import Agent

def get_weather(city: str) -> str:
    return f"{city}: 21°C"

agent = Agent(llm="lm_studio/qwen2.5-7b-instruct", tools=[get_weather])
agent.start("What is the weather in Paris?")

Every local prefix works the same way — swap the prefix, keep your tools:

from praisonaiagents import Agent

def get_weather(city: str) -> str:
    return f"{city}: 21°C"

# LM Studio
agent = Agent(llm="lm_studio/qwen2.5-7b-instruct", tools=[get_weather])

# vLLM (self-hosted)
agent = Agent(llm="vllm/meta-llama/Llama-3.1-8B-Instruct", tools=[get_weather])

# OpenAI-compatible vLLM (LiteLLM's hosted_vllm/ spelling)
agent = Agent(llm="hosted_vllm/Qwen2.5-7B", tools=[get_weather])

# llama.cpp
agent = Agent(llm="llamacpp/qwen2.5", tools=[get_weather])

Prefix	Server	Adapter	max_tool_repairs	Text-based tool-call recovery
lm_studio/…, lmstudio/…	LM Studio	LocalOpenAIAdapter	2	✅
vllm/…, hosted_vllm/…	vLLM	LocalOpenAIAdapter	2	✅
llamacpp/…, llama_cpp/…	llama.cpp llama-server	LocalOpenAIAdapter	2	✅
ollama_chat/…	Ollama (tool-calling)	OllamaAdapter	2	✅
ollama/…	Ollama	OllamaAdapter	2	✅
huggingface/…	Hosted Inference API (not local)	DefaultAdapter	0	❌
Hosted OpenAI-compatible via base_url	Hosted /v1 API	DefaultAdapter	0	❌

huggingface/… is the hosted Inference API, not a local server — it keeps the DefaultAdapter with max_tool_repairs=0 and is not part of the local prefix route.

Since PR #4870: llama_cpp/<model> is transparently rewritten to openai/<model> for LiteLLM (LiteLLM knows no llama_cpp provider). The LocalOpenAIAdapter is still selected — the adapter chooses on the original prefix. Before this fix the request retried four times with "LLM Provider NOT provided" and returned None.

Since PR #4870, a base_url ending in /v1 is not doubled — LM Studio and vLLM URLs work verbatim.

Local-server prefixes (lm_studio/, vllm/, hosted_vllm/, llamacpp/, llama_cpp/) also inherit text-based tool-call recovery: when the model emits a tool call as JSON text — common after a repair prompt on a small local model — the adapter parses it and dispatches the tool call anyway. A hosted OpenAI-compatible endpoint (a base_url pointing at a hosted API) deliberately does not get this behaviour: a hosted model returning JSON prose must never have it silently interpreted as a tool call. See Tool Call Self-Repair → Text-based tool-call recovery.

Explicit max_tool_repairs on the Agent always wins:

from praisonaiagents import Agent

agent = Agent(llm="lm_studio/qwen", max_tool_repairs=5)  # ends up as 5

Streaming with tools keeps working on LM Studio, vLLM, and llama.cpp — the LocalOpenAIAdapter deliberately does not inherit Ollama’s stream-disabling behaviour, because these servers stream tool calls correctly.

---

Choosing Which Style to Use

Pick the routing style that matches what you connect to.

---

Which Adapter Am I Getting?

A base_url says where the server is, not what it serves — so a local URL only implies Ollama for a model a local runtime could plausibly host.

---

Best Practices

Use base_url at the top level for one-off hosts

The top-level base_url is cleaner than the dict form for single-agent scripts. Reach for the dict form only when you want connection settings bundled inside llm.

Use env vars when the host is fixed

Set OPENAI_API_BASE and OPENAI_API_KEY when the host stays the same across runs. This keeps code portable across environments.

Keep embeddings on their own provider

Most OpenAI-compatible chat hosts don’t serve embeddings. Configure embeddings separately — see the Embeddings docs.

Don't add openai/ yourself

PraisonAI adds the openai/ prefix internally when base_url is set. Adding it manually is harmless but unnecessary.

Mixing base_url with hosted model names is safe

Closed-weights models — gpt-*, o1-*, o3-*, chatgpt-*, claude*, gemini-* — keep their native adapter regardless of base_url. Setting a local base_url (even http://localhost:11434/v1) no longer reassigns them to Ollama.

The exception is open-weights families — llama, gemma, mistral, qwen, deepseek, phi — which a local server can actually host. Behind a local base_url these are treated as Ollama, unlocking the tool-call repair budget. See Mixing local and hosted models for the full rules.

---

Related

OpenAI

First-party OpenAI models.

LiteLLM Proxy

Route through a self-hosted LiteLLM proxy gateway.

Ollama

Local Ollama models.

Custom Provider

Register a fully custom Python provider.