Rate Limiter

Cap API request rate and token usage across agents and threads

Cap LLM call rate so agents stay within provider quotas and budget — safely, even when many agents share one limiter.

from praisonaiagents import Agent, ExecutionConfig

agent = Agent(
    name="Researcher",
    instructions="Research topics concisely.",
    execution=ExecutionConfig(max_rpm=60),
)

agent.start("Summarise the latest Mars rover news")

The user runs agents at scale; the limiter queues or throttles LLM calls so quotas are not exceeded.

For bot/messaging rate limiting (Telegram, Discord, Slack), see Bot Rate Limiting. This page covers LLM API rate limiting.

How It Works

Quick Start

1. Simple Usage

   Set max_rpm on execution config for a single agent:

   from praisonaiagents import Agent, ExecutionConfig
   
   agent = Agent(
       name="Researcher",
       instructions="You research topics on the web.",
       execution=ExecutionConfig(max_rpm=60),
   )
   
   agent.start("Summarise the latest Mars rover news")
   
2. With Configuration

   Share one RateLimiter across multiple agents:

   from praisonaiagents import Agent, AgentTeam, ExecutionConfig
   from praisonaiagents.llm import RateLimiter
   
   shared = RateLimiter(requests_per_minute=60, burst=5)
   
   researcher = Agent(
       name="Researcher",
       instructions="Research topics",
       execution=ExecutionConfig(rate_limiter=shared),
   )
   writer = Agent(
       name="Writer",
       instructions="Write articles",
       execution=ExecutionConfig(rate_limiter=shared),
   )
   
   AgentTeam(agents=[researcher, writer]).start()
   

---

How It Works

Token bucket algorithm: tokens refill at requests_per_minute / 60 per second; each LLM call consumes one token. Under contention, callers wait until a token is available.

The limiter applies to both the initial LLM call and the follow-up after tool execution in streaming mode.

The rate limiter applies to every LLM call the agent issues — single-shot, streaming, tool-iteration and reflection turns, sync and async — so a per-model token budget is never spent on a path that bypasses throttling.

Step	What happens
Refill	Tokens regenerate on elapsed time
Acquire	Caller reserves a token (thread-safe)
Wait	Sleeps or awaits when bucket is empty
Release	Automatic — rolling window, no explicit release

---

Configuration Options

Option	Type	Default	Description
max_rpm	int	None	Shorthand on ExecutionConfig
requests_per_minute	int	None	Max requests per rolling 60s window
tokens_per_minute	int	None	Token budget for TPM-quoted providers
burst	int	1	Back-to-back requests before rate kicks in
max_retry_delay	int	120	Max wait seconds when rate limited

---

Best Practices

Share one limiter across related agents

When multiple agents use the same API key, pass the same RateLimiter so combined throughput stays in quota.

Match burst to your workload

Low burst (1–5) smooths traffic; higher burst tolerates spiky demand.

Set tokens_per_minute for TPM limits

Providers quote RPM and TPM — limiting only RPM can still trigger 429 errors.

Use async paths in async flows

agent.achat() calls acquire_async() automatically; avoid mixing sync and async limiters.

---

CLI

praisonai "task" --rpm 60

---

Related

Thread Safety

Thread-safe chat history and caches

Concurrency

Limit parallel agent runs