LLM Gateways

Route agent LLM calls through unified gateways like OpenRouter and LiteLLM Proxy for access to 100+ providers

Route agent LLM calls through OpenRouter, LiteLLM Proxy, or a custom OpenAI-compatible gateway — no manual registration required.

Gateway providers ship pre-registered. To register your own provider, see Custom Provider Registry — as of PR #3011 user-registered providers are visible to the same wrapper hot paths that gateway providers use.

import os
from praisonaiagents import Agent

if not os.getenv("OPENROUTER_API_KEY"):
    raise EnvironmentError("Set OPENROUTER_API_KEY in your environment")

agent = Agent(
    name="assistant",
    llm="openrouter/anthropic/claude-3.5-sonnet",
)
agent.start("Hello!")

The user picks a gateway-backed model string; the agent routes the request through the gateway to the underlying provider.

Quick Start

1. OpenRouter — one line

   import os
   from praisonaiagents import Agent
   
   if not os.getenv("OPENROUTER_API_KEY"):
       raise EnvironmentError("Set OPENROUTER_API_KEY in your environment")
   
   agent = Agent(
       name="assistant",
       llm="openrouter/anthropic/claude-3.5-sonnet",
   )
   agent.start("Hello!")
   

   Gateway providers auto-register on import — no register_gateway_providers() call needed.

2. LiteLLM Proxy

   import os
   from praisonaiagents import Agent
   
   os.environ["LITELLM_PROXY_BASE_URL"] = "http://localhost:4000"
   if not os.getenv("LITELLM_PROXY_API_KEY"):
       raise EnvironmentError("Set LITELLM_PROXY_API_KEY in your environment")
   
   agent = Agent(
       name="assistant",
       llm="litellm-proxy/gpt-4",
   )
   agent.start("Hello!")
   
3. Custom gateway (advanced)

   import os
   from praisonai.llm import create_llm_provider
   from praisonaiagents import Agent
   
   provider = create_llm_provider({
       "name": "custom-gateway",
       "model_id": "my-model",
       "config": {
           "base_url": "https://api.example.com/v1",
           "api_key": os.getenv("CUSTOM_GATEWAY_API_KEY", ""),
       },
   })
   agent = Agent(name="assistant", llm=provider)
   agent.start("Hello!")
   

---

Choose Your Gateway

Need	Gateway	Example llm
Hosted access to 100+ models	OpenRouter	openrouter/anthropic/claude-3.5-sonnet or or/gpt-4
Self-hosted caching, fallback, load balancing	LiteLLM Proxy	litellm-proxy/gpt-4
Internal OpenAI-compatible endpoint	Custom Gateway	custom-gateway/my-model with base_url in config

---

How It Works

---

Provider Reference

OpenRouter

Option	Type	Default	Description
api_key	str	$OPENROUTER_API_KEY	OpenRouter API key
base_url	str	https://openrouter.ai/api/v1	API endpoint
extra_headers	dict	—	e.g. X-Title, HTTP-Referer for app ranking

Aliases: openrouter, or. Model IDs are prefixed with openrouter/ automatically unless already prefixed.

LiteLLM Proxy

Option	Type	Default	Description
api_key	str	$LITELLM_PROXY_API_KEY	Proxy API key
base_url	str	$LITELLM_PROXY_BASE_URL or http://localhost:4000	Proxy URL
extra_headers	dict	—	Custom auth or routing headers

Aliases: litellm-proxy, llm-proxy, litellm-gateway. Model IDs are passed through unchanged.

Custom Gateway

Option	Type	Default	Description
base_url	str	required	OpenAI-compatible endpoint — raises ValueError if missing
api_key	str	—	Optional API key
extra_headers	dict	—	Custom headers

Aliases: custom-gateway, gateway, custom.

Explicit config["api_key"] overrides environment variables at init time.

---

Common Patterns

Multi-agent, different gateways:

from praisonaiagents import Agent

researcher = Agent(name="researcher", llm="openrouter/anthropic/claude-3.5-sonnet")
writer = Agent(name="writer", llm="litellm-proxy/gpt-4o-mini")

OpenRouter attribution headers:

from praisonai.llm import OpenRouterProvider
from praisonaiagents import Agent

provider = OpenRouterProvider("gpt-4", config={
    "extra_headers": {"X-Title": "My App", "HTTP-Referer": "https://myapp.com"},
})
agent = Agent(name="assistant", llm=provider)

Shorthand alias:

agent = Agent(name="assistant", llm="or/gpt-4")  # same as openrouter/gpt-4

---

Developer Flow

---

Best Practices

Prefer environment variables

Set OPENROUTER_API_KEY or LITELLM_PROXY_API_KEY instead of hardcoding keys in config.

OpenRouter for experimentation

Quick access to 100+ models with one API key — ideal for prototyping.

LiteLLM Proxy for production

Self-hosted caching, observability, and fallback routing suit production workloads.

Custom Gateway for compliance

Use CustomGatewayProvider when traffic must stay on an internal OpenAI-compatible proxy.

extra_headers for attribution

OpenRouter uses X-Title and HTTP-Referer for app ranking and routing hints.

---

Related

OpenRouter

OpenRouter model strings and env vars

LiteLLM Proxy

Self-hosted proxy setup

LLM Endpoint Config

Environment-based endpoint configuration

Custom Provider

Register custom LLM providers