> ## Documentation Index
> Fetch the complete documentation index at: https://praison.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# OpenAI-Compatible Endpoints

> Point PraisonAI at any /v1 host with base_url + api_key — no provider prefix needed

Point PraisonAI at any OpenAI-compatible `/v1` host by setting `base_url` and `api_key` on the `Agent` — bare catalog model ids are fine, no `provider/` prefix required.

<Tip>
  Running the server **locally**? If you don't want to hard-code the base URL or model name, use `llm="local"` — it discovers a running local server (Ollama, llama.cpp, LM Studio, vLLM, …) and configures itself. See the [Local Model Resolver](/docs/features/local-model-resolver).
</Tip>

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
graph LR
    subgraph "OpenAI-Compatible Routing"
        A[📝 Agent<br/>llm='deepseek-v4-flash'<br/>base_url='https://api.pzero.studio/v1'] --> R[🔌 Auto-prefix<br/>openai/deepseek-v4-flash]
        R --> H[🌐 /v1/chat/completions<br/>on your host]
        H --> O[✅ Response]
    end

    classDef input fill:#6366F1,stroke:#7C90A0,color:#fff
    classDef process fill:#F59E0B,stroke:#7C90A0,color:#fff
    classDef host fill:#189AB4,stroke:#7C90A0,color:#fff
    classDef output fill:#10B981,stroke:#7C90A0,color:#fff

    class A input
    class R process
    class H host
    class O output
```

## Quick Start

<Steps>
  <Step title="Simplest form">
    Pass `base_url` and `api_key` at the top level with a bare model id.

    ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    from praisonaiagents import Agent

    agent = Agent(
        name="assistant",
        instructions="You are a helpful assistant",
        llm="deepseek-v4-flash",
        base_url="https://api.pzero.studio/v1",
        api_key="your-key-or-empty-if-public",
    )
    agent.start("Explain what an OpenAI-compatible endpoint is in one paragraph.")
    ```
  </Step>

  <Step title="Environment-variable variant">
    Set the host once with environment variables, then keep the code clean.

    ```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    export OPENAI_API_BASE=https://api.pzero.studio/v1
    export OPENAI_API_KEY=your-key
    ```

    ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    from praisonaiagents import Agent

    agent = Agent(
        name="assistant",
        instructions="You are a helpful assistant",
        llm="deepseek-v4-flash",
    )
    agent.start("Hi")
    ```
  </Step>

  <Step title="Dict form (equivalent)">
    The dict form does the same thing when you prefer to bundle connection settings inside `llm`.

    ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    from praisonaiagents import Agent

    agent = Agent(
        name="assistant",
        instructions="You are a helpful assistant",
        llm={
            "model": "deepseek-v4-flash",
            "api_base": "https://api.pzero.studio/v1",
            "api_key": "",
        },
    )
    agent.start("Hello")
    ```
  </Step>
</Steps>

***

## How It Works

PraisonAI routes a bare model id plus a `base_url` through the OpenAI-compatible client.

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
sequenceDiagram
    participant User
    participant Agent
    participant Router as PraisonAI LLM
    participant Host as Your /v1 host

    User->>Agent: agent.start("...")
    Agent->>Router: model='deepseek-v4-flash', base_url set
    Note over Router: Bare model + base_url →<br/>route as openai/deepseek-v4-flash
    Router->>Host: POST /v1/chat/completions
    Host-->>Router: choices[0].message
    Router-->>Agent: text
    Agent-->>User: response
```

| Situation                                                   | What happens                               |
| ----------------------------------------------------------- | ------------------------------------------ |
| Bare model + `base_url` set                                 | Routed as `openai/<model>` to `<base_url>` |
| Prefixed model (e.g. `anthropic/…`, `ollama/…`, `openai/…`) | Passed through unchanged                   |
| Bare model with no `base_url` and no `OPENAI_API_BASE`      | Uses OpenAI default (`api.openai.com/v1`)  |

***

## Configuration Options

Set these top-level `Agent` parameters for any OpenAI-compatible host.

| Option     | Type            | Default  | Description                                                                               |
| ---------- | --------------- | -------- | ----------------------------------------------------------------------------------------- |
| `llm`      | `str` \| `dict` | required | Catalog model id (e.g. `"deepseek-v4-flash"`) or a full dict `{model, api_base, api_key}` |
| `base_url` | `str`           | `None`   | OpenAI-compatible `/v1` root, e.g. `https://api.pzero.studio/v1`                          |
| `api_key`  | `str`           | `None`   | API key for the host (may be empty for public catalogs)                                   |

<Card title="Agent SDK Reference" icon="code" href="/docs/api/praisonaiagents/agent/agent">
  Full parameter surface for the Agent class.
</Card>

***

## Common Patterns

Every host uses the same 5-line pattern — only `base_url` and the catalog id change.

<Tabs>
  <Tab title="P0 / pzero.studio">
    ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    from praisonaiagents import Agent

    agent = Agent(
        name="assistant",
        llm="deepseek-v4-flash",
        base_url="https://api.pzero.studio/v1",
        api_key="",
    )
    agent.start("Hello")
    ```
  </Tab>

  <Tab title="DeepInfra">
    ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    from praisonaiagents import Agent

    agent = Agent(
        name="assistant",
        llm="meta-llama/Meta-Llama-3-8B-Instruct",
        base_url="https://api.deepinfra.com/v1/openai",
        api_key="your-deepinfra-key",
    )
    agent.start("Hello")
    ```
  </Tab>

  <Tab title="Fireworks">
    ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    from praisonaiagents import Agent

    agent = Agent(
        name="assistant",
        llm="accounts/fireworks/models/llama-v3-8b-instruct",
        base_url="https://api.fireworks.ai/inference/v1",
        api_key="your-fireworks-key",
    )
    agent.start("Hello")
    ```
  </Tab>

  <Tab title="Together AI">
    ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    from praisonaiagents import Agent

    agent = Agent(
        name="assistant",
        llm="meta-llama/Llama-3-8b-chat-hf",
        base_url="https://api.together.xyz/v1",
        api_key="your-together-key",
    )
    agent.start("Hello")
    ```
  </Tab>

  <Tab title="vLLM">
    You can also use the `vllm/…` / `hosted_vllm/…` prefix directly — see [Local prefix route](/docs/models/openai-compatible#local-prefix-route-tool-call-repair) for the adapter with tool-call repair.

    ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    from praisonaiagents import Agent

    agent = Agent(
        name="assistant",
        llm="meta-llama/Meta-Llama-3-8B-Instruct",
        base_url="http://localhost:8000/v1",
        api_key="not-needed",
    )
    agent.start("Hello")
    ```
  </Tab>

  <Tab title="LM Studio">
    You can also use the `lm_studio/…` prefix directly — see [Local prefix route](/docs/models/openai-compatible#local-prefix-route-tool-call-repair) for the adapter with tool-call repair.

    ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    from praisonaiagents import Agent

    agent = Agent(
        name="assistant",
        llm="local-model",
        base_url="http://localhost:1234/v1",
        api_key="not-needed",
    )
    agent.start("Hello")
    ```
  </Tab>

  <Tab title="Company gateway">
    ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
    from praisonaiagents import Agent

    agent = Agent(
        name="assistant",
        llm="your-catalog-id",
        base_url="https://gateway.company.internal/v1",
        api_key="your-key",
    )
    agent.start("Hello")
    ```
  </Tab>
</Tabs>

***

## Local prefix route (tool-call repair)

For a locally-served OpenAI-compatible server — LM Studio, vLLM, llama.cpp's `llama-server` — use a dedicated prefix instead of `base_url + bare model`. PraisonAI recognises these prefixes and selects a `LocalOpenAIAdapter` that keeps the standard OpenAI message shape and adds a tool-call repair budget of 2 — a safety net for the malformed JSON tool calls small local models occasionally emit.

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents import Agent

def get_weather(city: str) -> str:
    return f"{city}: 21°C"

agent = Agent(llm="lm_studio/qwen2.5-7b-instruct", tools=[get_weather])
agent.start("What is the weather in Paris?")
```

Every local prefix works the same way — swap the prefix, keep your tools:

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents import Agent

def get_weather(city: str) -> str:
    return f"{city}: 21°C"

# LM Studio
agent = Agent(llm="lm_studio/qwen2.5-7b-instruct", tools=[get_weather])

# vLLM (self-hosted)
agent = Agent(llm="vllm/meta-llama/Llama-3.1-8B-Instruct", tools=[get_weather])

# OpenAI-compatible vLLM (LiteLLM's hosted_vllm/ spelling)
agent = Agent(llm="hosted_vllm/Qwen2.5-7B", tools=[get_weather])

# llama.cpp
agent = Agent(llm="llamacpp/qwen2.5", tools=[get_weather])
```

| Prefix                                  | Server                               | Adapter              | `max_tool_repairs` | Text-based tool-call recovery |
| --------------------------------------- | ------------------------------------ | -------------------- | ------------------ | ----------------------------- |
| `lm_studio/…`, `lmstudio/…`             | LM Studio                            | `LocalOpenAIAdapter` | 2                  | ✅                             |
| `vllm/…`, `hosted_vllm/…`               | vLLM                                 | `LocalOpenAIAdapter` | 2                  | ✅                             |
| `llamacpp/…`, `llama_cpp/…`             | llama.cpp `llama-server`             | `LocalOpenAIAdapter` | 2                  | ✅                             |
| `ollama_chat/…`                         | Ollama (tool-calling)                | `OllamaAdapter`      | 2                  | ✅                             |
| `ollama/…`                              | Ollama                               | `OllamaAdapter`      | 2                  | ✅                             |
| `huggingface/…`                         | **Hosted Inference API** (not local) | `DefaultAdapter`     | 0                  | ❌                             |
| Hosted OpenAI-compatible via `base_url` | Hosted `/v1` API                     | `DefaultAdapter`     | 0                  | ❌                             |

<Note>
  `huggingface/…` is the hosted Inference API, **not** a local server — it keeps the `DefaultAdapter` with `max_tool_repairs=0` and is not part of the local prefix route.
</Note>

<Note>
  **Since PR [#4870](https://github.com/MervinPraison/PraisonAI/pull/4870):** `llama_cpp/<model>` is transparently rewritten to `openai/<model>` for LiteLLM (LiteLLM knows no `llama_cpp` provider). The `LocalOpenAIAdapter` is still selected — the adapter chooses on the original prefix. Before this fix the request retried four times with `"LLM Provider NOT provided"` and returned `None`.
</Note>

<Note>
  Since PR #4870, a `base_url` ending in `/v1` is not doubled — LM Studio and vLLM URLs work verbatim.
</Note>

Local-server prefixes (`lm_studio/`, `vllm/`, `hosted_vllm/`, `llamacpp/`, `llama_cpp/`) also inherit **text-based tool-call recovery**: when the model emits a tool call as JSON text — common after a repair prompt on a small local model — the adapter parses it and dispatches the tool call anyway. A hosted OpenAI-compatible endpoint (a `base_url` pointing at a hosted API) deliberately does **not** get this behaviour: a hosted model returning JSON prose must never have it silently interpreted as a tool call. See [Tool Call Self-Repair → Text-based tool-call recovery](/docs/features/tool-call-self-repair#text-based-tool-call-recovery-local-models).

Explicit `max_tool_repairs` on the `Agent` always wins:

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents import Agent

agent = Agent(llm="lm_studio/qwen", max_tool_repairs=5)  # ends up as 5
```

Streaming with tools keeps working on LM Studio, vLLM, and llama.cpp — the `LocalOpenAIAdapter` deliberately does not inherit Ollama's stream-disabling behaviour, because these servers stream tool calls correctly.

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
graph TB
    Q{Local OpenAI-compatible server?} -->|Same host across many agents<br/>or non-standard /v1 path| Base[Use base_url + bare model]
    Q -->|One-off script + tool calling<br/>on a small local model| Prefix[Use lm_studio/… vllm/… llamacpp/…<br/>gets LocalOpenAIAdapter + repair budget]

    classDef q fill:#6366F1,stroke:#7C90A0,color:#fff
    classDef opt fill:#189AB4,stroke:#7C90A0,color:#fff

    class Q q
    class Base,Prefix opt
```

***

## Choosing Which Style to Use

Pick the routing style that matches what you connect to.

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
graph TB
    Q{What are you connecting to?} -->|Named provider<br/>OpenAI, Anthropic, Gemini| Named[Use provider prefix<br/>e.g. anthropic/claude-sonnet-4-5]
    Q -->|OpenAI-compatible /v1 host| Compat{Local server?<br/>LM Studio · vLLM · llama.cpp}
    Compat -->|Generic host| Base[Use base_url + bare model]
    Compat -->|Local server + tool calling| Local[Use local prefix<br/>lm_studio/… vllm/… llamacpp/…]
    Q -->|Self-hosted LiteLLM Proxy<br/>as a gateway| Proxy[Use litellm-proxy/model<br/>with LITELLM_PROXY_* env]
    Q -->|Custom Python provider class| Reg[Register in Custom Provider Registry]

    classDef q fill:#6366F1,stroke:#7C90A0,color:#fff
    classDef opt fill:#189AB4,stroke:#7C90A0,color:#fff

    class Q,Compat q
    class Named,Base,Local,Proxy,Reg opt
```

***

## Which Adapter Am I Getting?

A `base_url` says *where* the server is, not *what* it serves — so a local URL only implies Ollama for a model a local runtime could plausibly host.

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
graph TB
    Start[📝 model + base_url] --> Prefix{ollama/ prefix?}
    Prefix -->|Yes| Ollama[✅ OllamaAdapter]
    Prefix -->|No| Hosted{Closed-weights name?<br/>gpt-* o1-* claude* gemini-*}
    Hosted -->|Yes| Native[✅ Native adapter<br/>Default · Anthropic · Gemini]
    Hosted -->|No| LocalUrl{Local base_url?<br/>ollama or :11434}
    LocalUrl -->|Yes| Ollama
    LocalUrl -->|No| Name{claude / gemini<br/>in name?}
    Name -->|Yes| Native
    Name -->|No| Default[✅ DefaultAdapter]

    classDef input fill:#8B0000,stroke:#7C90A0,color:#fff
    classDef decision fill:#6366F1,stroke:#7C90A0,color:#fff
    classDef warn fill:#F59E0B,stroke:#7C90A0,color:#fff
    classDef output fill:#10B981,stroke:#7C90A0,color:#fff

    class Start input
    class Prefix,Hosted,LocalUrl,Name decision
    class Ollama warn
    class Native,Default output
```

***

## Best Practices

<AccordionGroup>
  <Accordion title="Use base_url at the top level for one-off hosts">
    The top-level `base_url` is cleaner than the dict form for single-agent scripts. Reach for the dict form only when you want connection settings bundled inside `llm`.
  </Accordion>

  <Accordion title="Use env vars when the host is fixed">
    Set `OPENAI_API_BASE` and `OPENAI_API_KEY` when the host stays the same across runs. This keeps code portable across environments.
  </Accordion>

  <Accordion title="Keep embeddings on their own provider">
    Most OpenAI-compatible chat hosts don't serve embeddings. Configure embeddings separately — see the [Embeddings](/docs/capabilities/embeddings) docs.
  </Accordion>

  <Accordion title="Don't add openai/ yourself">
    PraisonAI adds the `openai/` prefix internally when `base_url` is set. Adding it manually is harmless but unnecessary.
  </Accordion>

  <Accordion title="Mixing base_url with hosted model names is safe">
    Closed-weights models — `gpt-*`, `o1-*`, `o3-*`, `chatgpt-*`, `claude*`, `gemini-*` — keep their native adapter regardless of `base_url`. Setting a local `base_url` (even `http://localhost:11434/v1`) no longer reassigns them to Ollama.

    The exception is open-weights families — `llama`, `gemma`, `mistral`, `qwen`, `deepseek`, `phi` — which a local server can actually host. Behind a local `base_url` these are treated as Ollama, unlocking the tool-call repair budget. See [Mixing local and hosted models](/docs/features/local-models#mixing-local-and-hosted-models) for the full rules.
  </Accordion>
</AccordionGroup>

***

## Related

<CardGroup cols={2}>
  <Card title="OpenAI" icon="robot" href="/docs/models/openai">
    First-party OpenAI models.
  </Card>

  <Card title="LiteLLM Proxy" icon="server" href="/docs/models/litellm-proxy">
    Route through a self-hosted LiteLLM proxy gateway.
  </Card>

  <Card title="Ollama" icon="microchip" href="/docs/models/ollama">
    Local Ollama models.
  </Card>

  <Card title="Custom Provider" icon="wrench" href="/docs/models/custom-provider">
    Register a fully custom Python provider.
  </Card>
</CardGroup>