Failover

Automatic fallback between LLM providers for reliability and cost optimization

Failover automatically switches between LLM providers when one fails, keeping agents running during API outages or rate limits.

import os
from praisonaiagents import Agent, AuthProfile, FailoverConfig, FailoverManager

manager = FailoverManager(FailoverConfig(max_retries=3))
manager.add_profile(AuthProfile(
    name="openai", provider="openai",
    api_key=os.getenv("OPENAI_API_KEY"), priority=1,
))
manager.add_profile(AuthProfile(
    name="anthropic", provider="anthropic",
    api_key=os.getenv("ANTHROPIC_API_KEY"), priority=2,
))

agent = Agent(name="assistant", llm={"model": "gpt-4o-mini", "failover_manager": manager})
agent.start("Hello!")

The user sends a message; if the primary LLM profile fails, the agent tries the next profile in the failover chain.

Failover integrates with LLM Error Classification via FailoverDecision, which coordinates profile rotation with typed error handling.

Quick Start

1. Run an agent with failover

   import os
   from praisonaiagents import Agent, AuthProfile, FailoverConfig, FailoverManager
   
   manager = FailoverManager(FailoverConfig(max_retries=3, exponential_backoff=True))
   manager.add_profile(AuthProfile(
       name="openai", provider="openai",
       api_key=os.getenv("OPENAI_API_KEY"), priority=1,
   ))
   manager.add_profile(AuthProfile(
       name="anthropic", provider="anthropic",
       api_key=os.getenv("ANTHROPIC_API_KEY"), priority=2,
   ))
   
   agent = Agent(
       name="assistant",
       llm={"model": "gpt-4o-mini", "failover_manager": manager},
   )
   agent.start("Hello!")
   
2. Add more profiles

   import os
   from praisonaiagents import AuthProfile, FailoverManager
   
   manager = FailoverManager()
   manager.add_profile(AuthProfile(
       name="groq", provider="groq",
       api_key=os.getenv("GROQ_API_KEY"), priority=3,
   ))
   
3. Monitor provider health

   status = manager.status()
   for name, info in status.items():
       print(f"{name}: {info['status']}")
   

---

How failover activates during retries

Failover now drives LLM retries through direct integration with the retry mechanism:

• On every LLM call, the system first gets the current profile via get_next_profile() and applies its api_key, base_url, and model settings
• On success, mark_success(profile) is called to track the working provider
• On failure, mark_failure(profile, error, is_rate_limit=...) marks the provider as failed, then get_next_profile() fetches the next available provider
• Profile switching overrides non-retryable classification—one extra attempt is always granted after switching providers
• The LLM automatically updates request parameters (api_key, base_url, model) when switching between profiles

---

How It Works

Component	Role
AuthProfile	Credentials for a single provider
FailoverManager	Orchestrates failover logic
FailoverConfig	Retry and backoff settings
ProviderStatus	Tracks provider health

---

Configuration Options

FailoverManager

Manager class reference

AuthProfile

Provider credential profile

from praisonaiagents import FailoverConfig

config = FailoverConfig(
    max_retries=3,
    retry_delay=1.0,
    exponential_backoff=True,
    max_retry_delay=60.0,
)

Option	Type	Default	Description
max_retries	int	3	Maximum retry attempts
retry_delay	float	1.0	Initial retry delay
exponential_backoff	bool	True	Use exponential backoff
max_retry_delay	float	60.0	Maximum retry delay
cooldown_on_rate_limit	float	60.0	Rate limit cooldown (seconds)
cooldown_on_error	float	30.0	Error cooldown (seconds)
rotate_on_success	bool	False	Rotate profiles on success

---

Auth Profiles

Configure credentials for each provider:

import os
from praisonaiagents import AuthProfile

profile = AuthProfile(
    name="openai-primary",
    provider="openai",
    api_key=os.getenv("OPENAI_API_KEY"),
    priority=1,
    rate_limit_rpm=100,
)

Field	Type	Description
name	str	Unique profile identifier
provider	str	Provider: openai, anthropic, etc.
api_key	str	API key (masked in logs)
base_url	str	Custom API endpoint
model	str	Default model for this profile
priority	int	Failover priority (lower = higher priority)
rate_limit_rpm	int	Requests per minute limit
rate_limit_tpm	int	Tokens per minute limit
metadata	dict	Additional provider-specific config

---

Common Patterns

Multi-Provider

from praisonaiagents import AuthProfile, FailoverManager

manager = FailoverManager()

# Add multiple providers
manager.add_profile(AuthProfile(
    name="openai", provider="openai",
    api_key=os.getenv("OPENAI_API_KEY"), priority=1,
))

manager.add_profile(AuthProfile(
    name="anthropic", provider="anthropic",
    api_key=os.getenv("ANTHROPIC_API_KEY"), priority=2,
))

manager.add_profile(AuthProfile(
    name="groq", provider="groq",
    api_key=os.getenv("GROQ_API_KEY"), priority=3,
))

Cost Optimization

import os
from praisonaiagents import AuthProfile, FailoverManager

manager = FailoverManager()
manager.add_profile(AuthProfile(
    name="gpt-4o-mini", provider="openai",
    api_key=os.getenv("OPENAI_API_KEY"), priority=1,
    model="gpt-4o-mini",
))
manager.add_profile(AuthProfile(
    name="gpt-4o", provider="openai",
    api_key=os.getenv("OPENAI_API_KEY"), priority=2,
    model="gpt-4o",
))

Regional Failover

import os
from praisonaiagents import AuthProfile, FailoverManager

manager = FailoverManager()
manager.add_profile(AuthProfile(
    name="openai-us", provider="openai",
    api_key=os.getenv("OPENAI_API_KEY"),
    base_url="https://api.openai.com/v1", priority=1,
))
manager.add_profile(AuthProfile(
    name="azure-eu", provider="azure",
    api_key=os.getenv("AZURE_OPENAI_API_KEY"),
    base_url=os.getenv("AZURE_OPENAI_ENDPOINT"), priority=2,
))

---

Failover Callbacks

React to failover events:

from praisonaiagents import FailoverManager, FailoverConfig

def on_failover(from_profile, to_profile, error):
    print(f"Failing over from {from_profile} to {to_profile}")
    print(f"Reason: {error}")
    # Log to monitoring system
    
config = FailoverConfig(
    on_failover=on_failover
)

manager = FailoverManager(config)

Sync + async parity (from PraisonAI #2386). The retry callback now fires on both the sync (agent.chat) and async (await agent.achat) LLM paths. Older versions silently skipped the callback on async calls — upgrade to get retry observability for gateway bots, async tools, and any await-based agent code.

---

Provider Status

Monitor provider health:

from praisonaiagents import FailoverManager

manager = FailoverManager()

# Get status of all providers
status = manager.status()
for name, info in status.items():
    print(f"{name}: {info['status']}")
    print(f"  Failures: {info['failure_count']}")
    print(f"  Last success: {info['last_success']}")

# Reset a provider after recovery
manager.mark_success("openai")

# Reset all profiles
manager.reset_all()

---

Best Practices

Configure multiple providers

Always have at least 2-3 providers configured. This ensures availability even during major outages.

Use exponential backoff

Enable exponential_backoff=True to avoid hammering providers during issues. This helps you stay within rate limits.

Set appropriate priorities

Order providers by cost and reliability. Put cheaper/faster providers first, with premium providers as fallback.

Monitor failover events

Use the on_failover callback to track when failovers occur. This helps identify provider issues early. The callback fires on both sync and async LLM paths — works the same whether your agent runs via agent.chat() or await agent.achat().

Integrate with error classification

Pair failover with LLM Error Classification so FailoverDecision coordinates profile rotation with typed errors.

Keep API keys out of source

Load keys from environment variables or a secrets manager — never commit credentials to version control.

---

Related

LLM Error Classification

Typed errors that drive failover decisions

Providers

Supported LLM providers