Tool Retry Policy

Automatically retry failed tool calls with exponential backoff

Tool retry automatically re-runs a failing tool with exponential backoff so transient errors don’t break your agent.

from praisonaiagents import Agent, tool
from praisonaiagents.config.feature_configs import ToolConfig
from praisonaiagents.tools import RetryPolicy

@tool
def web_search(query: str) -> str:
    """Search the web."""
    return f"Results for: {query}"

agent = Agent(
    name="researcher",
    instructions="Research topics on the web",
    tools=[web_search],
    tool_config=ToolConfig(retry_policy=RetryPolicy()),
)
agent.start("Find information about renewable energy")

The user asks for web research; RetryPolicy re-runs retryable tool errors with backoff before surfacing a failure.

Quick Start

1. Enable with defaults

   Enable retry for all tools with safe defaults:

   from praisonaiagents import Agent
   from praisonaiagents.config.feature_configs import ToolConfig
   from praisonaiagents.tools import RetryPolicy
   
   agent = Agent(
       name="researcher",
       instructions="Research topics on the web",
       tools=[web_search],
       tool_config=ToolConfig(retry_policy=RetryPolicy())  # 3 attempts, exponential backoff
   )
   
   agent.start("Find information about renewable energy")
   
2. Tune attempts and backoff

   Configure specific retry behavior:

   from praisonaiagents import Agent
   from praisonaiagents.config.feature_configs import ToolConfig
   from praisonaiagents.tools import RetryPolicy
   
   agent = Agent(
       name="api_agent", 
       instructions="Call external APIs",
       tools=[api_tool],
       tool_config=ToolConfig(
           retry_policy=RetryPolicy(
               max_attempts=5,
               retry_on={"timeout", "rate_limit", "connection_error"},
               backoff_factor=2.0,
               initial_delay_ms=1000,
               jitter=True,
           ),
       )
   )
   
3. Override per tool

   Different tools may need different retry strategies:

   from praisonaiagents import Agent, tool
   from praisonaiagents.config.feature_configs import ToolConfig
   from praisonaiagents.tools import RetryPolicy
   
   @tool(retry_policy=RetryPolicy(max_attempts=5, backoff_factor=3.0))
   def unreliable_api_call(query: str) -> str:
       """Call an unreliable external API."""
       # This tool gets aggressive retry policy
       return call_external_api(query)
   
   agent = Agent(
       name="mixed_agent",
       tools=[local_tool, unreliable_api_call],
       tool_config=ToolConfig(retry_policy=RetryPolicy(max_attempts=2))  # Default for other tools
   )
   

---

How It Works

Step	What happens
1	Agent calls tool via _execute_tool_with_circuit_breaker (sync) or execute_tool_async (async)
2	On error — a returned error string or a raised exception — _classify_error_type tags it: timeout, rate_limit, connection_error, or unknown. On the async path the flattened error dict carries the framework’s private _praison_retryable verdict, which the retry loop reads and then strips before returning — so it never reaches the model nor collides with a tool’s own retryable field
3	_effective_tool_retry_policy resolves the one active policy (per-tool > agent ToolConfig > translated ExecutionConfig) — both retry loops share it
4	If policy.should_retry(error_type, attempt) is true, wait policy.get_delay_ms(attempt) ms and retry
5	HookEvent.ON_RETRY fires before each retry with the new OnRetryInput fields

An error is tagged rate_limit when its message contains either "rate…limit" or "too many requests" — the latter catches HTTP 429 responses whose bodies use the standard phrasing without the word “limit”:

# Both of these classify as rate_limit and are retried by default:
raise Exception("Rate limit exceeded")
raise Exception("HTTP 429: Too Many Requests")

---

Sync / async parity

RetryPolicy applies identically on the sync path (agent.execute_tool(...), agent.start(...)) and the async path (agent.execute_tool_async(...), agent.achat(...)) — including tools that raise exceptions.

Fixed in PR #4141. Before that release, agent.achat(...) would silently not retry a raised exception, even when ToolConfig(retry_policy=RetryPolicy(max_attempts=N)) was set. The sync path already retried; the async path returned on the first attempt.

Parity also fixed for the restart_safe=False veto in PR #4299 (issue #4298). PR #4141 restored async retry for raised exceptions; PR #4299 restored async veto for tools that declared themselves unsafe. Before #4299, agent.achat() / agent.execute_tool_async() re-ran a @tool(restart_safe=False) body up to max_attempts times even though the sync path already honoured the declaration.

On the sync path a tool that exhausts its per-tool retries raises ToolExecutionError, which propagates directly out of agent.chat() / agent.start() (fix in PR #4252). It is not routed through the LLM error classifier — so a tool message that looks like a rate limit (e.g. "429 too many requests") does not trigger an LLM backoff time.sleep or a redundant model call. The per-tool retry wrapper (max_attempts) is what governs tool-error retries; retry / RetryBackoffConfig are LLM-error-driven and never fire on a tool failure. See Error Handling → Tool Failure Behavior.

import asyncio
from praisonaiagents import Agent
from praisonaiagents.config.feature_configs import ToolConfig
from praisonaiagents.tools import RetryPolicy

calls = []

def flaky():
    """Simulate a transient failure."""
    calls.append(1)
    raise RuntimeError("temporary glitch")

agent = Agent(
    instructions="Use the flaky tool.",
    tools=[flaky],
    tool_config=ToolConfig(retry_policy=RetryPolicy(max_attempts=3, initial_delay_ms=0)),
)

# Sync path — retries 3 times
try: agent.execute_tool("flaky", {})
except Exception: pass

# Async path — also retries 3 times (parity restored in PR #4141)
try: asyncio.run(agent.execute_tool_async("flaky", {}))
except Exception: pass

The veto is now honoured on both paths too (PR #4299) — a declared-unsafe tool runs exactly once whether called sync or async:

import asyncio
from praisonaiagents import Agent, tool
from praisonaiagents.config.feature_configs import ToolConfig
from praisonaiagents.tools import RetryPolicy

calls = []

@tool(restart_safe=False)
def charge_card(amount: str) -> str:
    """Charge a card — must never be silently re-driven."""
    calls.append(amount)
    raise RuntimeError("gateway 502")

agent = Agent(
    instructions="Charge the customer.",
    tools=[charge_card],
    tool_config=ToolConfig(retry_policy=RetryPolicy(max_attempts=3, initial_delay_ms=0)),
)

# Sync path — runs once (was already correct)
try: agent.execute_tool("charge_card", {"amount": "100"})
except Exception: pass
assert len(calls) == 1

calls.clear()
# Async path — also runs once (fixed in PR #4299)
try: asyncio.run(agent.execute_tool_async("charge_card", {"amount": "100"}))
except Exception: pass
assert len(calls) == 1

Both paths share the same terminal rule for raised exceptions:

Exception raised by tool	Retried?
ValueError, TypeError, AttributeError	terminal — programming errors
ToolExecutionError(is_retryable=False) (e.g. Loop Guard HALT)	terminal — explicit verdict honoured
ToolExecutionError(is_retryable=True)	retried up to max_attempts
Any other exception (e.g. RuntimeError, network error)	retried up to max_attempts

On the async path a raised exception is flattened into an error dict carrying the private _praison_retryable verdict, then the retry loop honours it and strips it before returning:

---

Your tool’s own retryable field is safe

Many HTTP-API wrappers return payloads shaped like {"error": "...", "retryable": true}. That field is the tool’s payload and is preserved untouched on the surfaced result. The framework’s own verdict lives on the private _praison_retryable key, which is stripped before the result reaches the model or the caller — the two never collide.

To make a returned error retry, tag its error type via RetryPolicy(retry_on=...). A word in the tool’s payload is not how retry is controlled.

from praisonaiagents import Agent, tool

@tool
def call_upstream(endpoint: str) -> dict:
    """Return a payload that includes its own retryable field."""
    return {"error": "upstream 503", "retryable": True}

agent = Agent(name="api", tools=[call_upstream])

result = agent.execute_tool("call_upstream", {"endpoint": "/status"})

assert result.get("retryable") is True        # tool payload survives
assert "_praison_retryable" not in result      # framework control key stripped

The same holds on the async path — the body still runs exactly once:

import asyncio
from praisonaiagents import Agent, tool

@tool
def call_upstream(endpoint: str) -> dict:
    """Return a payload that includes its own retryable field."""
    return {"error": "upstream 503", "retryable": True}

agent = Agent(name="api", tools=[call_upstream])

result = asyncio.run(agent.execute_tool_async("call_upstream", {"endpoint": "/status"}))

assert result.get("retryable") is True
assert "_praison_retryable" not in result

---

Returned error dicts run once

A tool that returns an error dict has already reached its own decision — the body ran to completion. Re-running it would duplicate side effects (charge money, send a message, write a row) for a decision the tool already made. The sync outer loop therefore surfaces such a dict exactly once, regardless of what fields it contains.

To make a returned error retry-eligible, register its error type via RetryPolicy(retry_on={...}) — the inner loop honours that classification, and the outer loop will not double-drive it.

---

Outer wall-clock timeouts retry only for idempotent tools

The outer loop’s own wall-clock timeout (_outer_timeout) is now gated on the tool’s idempotency: a timed-out tool is re-driven only if it is classified as idempotent. The timed-out body is still running in an orphaned worker thread (Future.cancel() is a no-op once started), so auto-retrying a mutating tool would duplicate its side effects. Idempotency comes from an explicit idempotent attribute on the tool, the global registry (with ToolConfig(allow_global_tools=True)), or the shared IDEMPOTENT_TOOLS / MUTATING_TOOLS name registry.

MUTATING_TOOLS gates timeout-retry decisions here; it does not veto ordinary tool-error retries. See Tool Retry & Backoff → Retrying Non-Idempotent Tools for how the veto works on the ordinary retry path.

Unknown tools default to non-idempotent (safe): a wall-clock timeout on an unrecognised tool surfaces once instead of retrying. See Tool Timeouts → Retry-on-timeout is opt-in for the .idempotent = True opt-in.

---

RetryPolicy is the single budget

The effective RetryPolicy is the one budget for tool-body runs. RetryPolicy.max_attempts genuinely caps how many times a tool runs, and the ExecutionConfig spelling (max_retry_limit / retry_*) is translated into that same policy in one place.

from praisonaiagents import Agent, tool
from praisonaiagents.config.feature_configs import ToolConfig
from praisonaiagents.tools import RetryPolicy

@tool
def charge_card(amount: int) -> str:
    """Charge a payment — must run at most once."""
    return f"Charged {amount}"

# Run a non-idempotent tool exactly once — no duplicate charges
agent = Agent(
    name="billing",
    tools=[charge_card],
    tool_config=ToolConfig(retry_policy=RetryPolicy(max_attempts=1)),
)
agent.start("Charge the customer $50")

Set RetryPolicy(max_attempts=1) for non-idempotent tools — a payment, an email, a POST. This now runs the tool exactly once, preventing duplicate side effects.

ExecutionConfig.retry_* and ToolConfig(retry_policy=...) are two spellings of one budget, not two separate loops. See Tool Retry & Backoff and ExecutionConfig for the ExecutionConfig spelling.

How many times a failing tool body actually runs:

Configuration	Tool-body runs
ExecutionConfig(max_retry_limit=0)	1×
ExecutionConfig(max_retry_limit=2)	3×
ToolConfig(retry_policy=RetryPolicy(max_attempts=1))	1×
ToolConfig(retry_policy=RetryPolicy(max_attempts=2))	2×

---

Precedence Ladder

When no retry_policy is set, the fallback is the policy translated from ExecutionConfig (max_retry_limit / retry_*), which defaults to RetryPolicy(max_attempts=3) when ExecutionConfig is untouched. Users who never set retry_policy= see no behaviour change — the translated policy reproduces max_retry_limit + 1 attempts and the same backoff.

retry_policy=None on @tool(...) means use the fallback — the tool-level slot is treated as empty, not as “no retries”. To disable retries for a specific tool, pass an explicit RetryPolicy(max_attempts=1) instead.

Tool-level (highest priority):

@tool(retry_policy=RetryPolicy(max_attempts=5))
def flaky_api():
    pass

Do NOT pass retry_policy=None to disable retries. @tool(retry_policy=None) is treated as “no policy set here” and falls through to the agent-level or default RetryPolicy. To actually disable retries for a specific tool, pass RetryPolicy(max_attempts=1).

# ✅ Disable retries for a specific tool
@tool(retry_policy=RetryPolicy(max_attempts=1))
def one_shot_only():
    ...

# ⚠️ retry_policy=None falls through to agent/default policy — NOT "no retries"
@tool(retry_policy=None)
def uses_agent_or_default():
    ...

Agent-level (medium priority):

from praisonaiagents import Agent
from praisonaiagents.config.feature_configs import ToolConfig
from praisonaiagents.tools import RetryPolicy

agent = Agent(tool_config=ToolConfig(retry_policy=RetryPolicy(max_attempts=3)))

Translated ExecutionConfig fallback (lowest priority):

from praisonaiagents import Agent, ExecutionConfig

# max_retry_limit / retry_* are translated into the effective RetryPolicy.
# Untouched ExecutionConfig → RetryPolicy(max_attempts=3).
agent = Agent(tools=[tool], execution=ExecutionConfig(max_retry_limit=4))

Task-scoped tools

A tool passed to a task via tools_override (not registered on the agent) participates in the same idempotency lookup as an agent-level tool. Its restart_safe=False / idempotent=False declaration is honoured on both sync and async paths (PR #4299). Task-scoped tools shadow same-named agent tools when both are present, so a safer per-task override wins.

from praisonaiagents import Agent, tool

@tool(restart_safe=False)
def charge_card(amount: str) -> str:
    """Charge a customer — the per-task safe version, wins over any agent-level tool."""
    ...

agent = Agent(instructions="Billing.", tools=[])  # no agent-level tools
await agent.execute_tool_async(
    "charge_card", {"amount": "100"}, tools_override=[charge_card]
)  # runs exactly once

---

Choosing a Retry Policy

---

Configuration Options

Option	Type	Default	Description
max_attempts	int	3	Total attempts including the first try
initial_delay_ms	int	1000	Delay before first retry, in milliseconds
backoff_factor	float	2.0	Multiplier applied to delay per attempt
retry_on	set[str]	{"timeout","rate_limit","connection_error"}	Error types that trigger a retry. rate_limit matches messages containing "rate…limit" or "too many requests"
jitter	bool	False	Add randomized jitter to delays
jitter_factor	float	0.25	Jitter range as fraction of delay (±25%)
max_delay_ms	int	30000	Maximum delay between retries

Non-retryable error types (always short-circuit):

• approval_denied, permission_denied, approval_error, circuit_open, loop_blocked
• Python exceptions: ValueError, TypeError, AttributeError from tool code
• Raised ValueError / TypeError / AttributeError from tool code are terminal on both sync and async paths.
• Argument-binding errors (missing/extra parameters resolved from the schema) are terminal — they never retry regardless of policy.

A loop_blocked result comes from Loop Guard. Bumping max_attempts cannot bypass it — configure the loop-guard thresholds instead.

---

Common Patterns

Per-tool override for unreliable API

from praisonaiagents import Agent
from praisonaiagents.config.feature_configs import ToolConfig
from praisonaiagents.tools import RetryPolicy

@tool(retry_policy=RetryPolicy(
    max_attempts=5,
    backoff_factor=3.0,
    jitter=True,
    retry_on={"timeout", "rate_limit", "connection_error"}
))
def external_weather_api(location: str) -> str:
    """Get weather from external API - known to be flaky."""
    return requests.get(f"https://api.weather.com/current?q={location}").text

agent = Agent(
    name="weather_bot",
    tools=[external_weather_api, local_calculation],
    tool_config=ToolConfig(retry_policy=RetryPolicy(max_attempts=2))  # Default for other tools
)

YAML configuration

In YAML the field name is still tool_retry_policy:; in Python pass retry settings through tool_config=ToolConfig(retry_policy=…). The standalone tool_retry_policy kwarg on Agent(...) was removed and raises TypeError.

agents:
  api_researcher:
    role: API Researcher
    instructions: "Research using external APIs"
    tools: [web_search, api_tool]
    tool_retry_policy:
      max_attempts: 4
      retry_on: [timeout, rate_limit]
      backoff_factor: 2.0
      jitter: true

CLI usage

praisonai \
  --tool-retry-attempts 5 \
  --tool-retry-delay 500 \
  --tool-retry-backoff 2.0 \
  --tool-retry-on "timeout,rate_limit" \
  "Research renewable energy trends"

If the retry backend isn’t available (the CLI accepted the flags but the runtime dependency is missing), you’ll now see a tool_retry_policy requested but retry backend unavailable: <ImportError> warning in the logs instead of the settings being silently dropped. Install the retry backend or drop the --tool-retry-* flags to clear the warning.

---

Hook Integration

Monitor retry attempts with hooks:

from praisonaiagents import Agent
from praisonaiagents.config.feature_configs import ToolConfig
from praisonaiagents.hooks import add_hook, HookEvent, HookResult, OnRetryInput
from praisonaiagents.tools import RetryPolicy

@add_hook(HookEvent.ON_RETRY)
def log_retry(event: OnRetryInput) -> HookResult:
    print(f"[retry] {event.tool_name} attempt {event.attempt}/{event.max_attempts} "
          f"after {event.delay_ms}ms — {event.error_type}: {event.error}")
    return HookResult.allow()

agent = Agent(
    name="monitored_agent",
    tools=[flaky_tool],
    tool_config=ToolConfig(retry_policy=RetryPolicy(max_attempts=3)),
)

Available fields on OnRetryInput:

• tool_name: Name of the failing tool
• attempt: Current attempt number (1-based)
• max_attempts: Maximum attempts configured
• delay_ms: Delay before this retry in milliseconds
• error_type: Classified error type (timeout, rate_limit, etc.)
• error: Original exception object

---

Best Practices

Keep max_attempts small (3-5)

Large retry counts mask real failures. If a tool fails 10+ times, there’s likely a deeper issue that retrying won’t solve. Use monitoring instead.

Always set jitter=True for rate-limited APIs

Without jitter, multiple agents retrying simultaneously create a “thundering herd” that can overwhelm rate-limited services. Jitter spreads out retry attempts.

Set narrower retry_on for expensive tools

Don’t retry LLM tools on connection_error if every attempt costs money. Use specific error types that indicate transient failures.

expensive_llm_tool_policy = RetryPolicy(
    max_attempts=2,
    retry_on={"timeout"}  # Only timeout, not connection errors
)

Use tool-level override sparingly

Agent-level retry policy keeps configuration DRY. Only override at the tool level for genuinely special cases like unreliable third-party APIs.

---

Related

Tool Configuration

Consolidated tool configuration with ToolConfig

Concurrency

Parallel tool execution and timeouts

Hooks

Monitor and intercept agent behavior

Hook Events

Complete reference of hook events