Prompt Cache Optimization

Stable, cache-friendly prompts across memory, tools, and tasks for lower cost and latency

Turn on memory on a supported model — the SDK sorts tools and lays out context so providers can cache the stable prefix.

from praisonaiagents import Agent

agent = Agent(
    instructions="You are a helpful assistant",
    llm="openai/gpt-4o",
    memory=True,
)

agent.start("Summarise the latest report")
# Turn 2+ reuses cached system prompt + memory prefix
agent.start("What changed since yesterday?")

The user chats across turns; stable prefixes stay cached for lower cost.

How It Works

Quick Start

1. Simple Usage

   Automatic on supported models when memory is enabled:

   from praisonaiagents import Agent
   
   agent = Agent(
       instructions="You are a helpful assistant",
       llm="openai/gpt-4o",
       memory=True,
   )
   
   agent.start("Summarise the latest report")
   
2. With Configuration

   Make prompt caching explicit:

   from praisonaiagents import Agent, CachingConfig
   
   agent = Agent(
       instructions="You are a helpful assistant",
       llm="anthropic/claude-sonnet-4-20250514",
       memory=True,
       caching=CachingConfig(prompt_caching=True),
   )
   
   agent.start("Analyse the data trends")
   

---

How It Works

Behaviour	What it does
Deterministic tool order	Tools sorted by function name — reordering tools does not break cache
Stable memory prefix	Memory sections in fixed order via build_cache_optimized_context()
Reserved boundary constant	CACHE_BOUNDARY is defined for future structured-cache metadata — not injected into current prompts

CACHE_BOUNDARY is a reserved marker constant ("--- CACHE_BOUNDARY ---") on praisonaiagents.memory.memory for future structured-cache metadata. No current provider consumes it, so it is intentionally not appended to the outgoing system prompt. Providers hit the cache because the prefix is deterministic — not because of any marker.

On OpenAI and Gemini the SDK emits no explicit cache markers. Those providers apply prefix caching automatically as long as the request prefix is byte-identical across turns. The optimisations on this page — deterministic tool order, stable memory layout, no wall-clock in the system prompt — are what make automatic caching work.

Anthropic works differently: it gets explicit cache_control breakpoints on the system block and the stable history prefix. See Prompt Caching for the Anthropic breakpoint behaviour and the provider-behaviour table.

Check model support:

from praisonaiagents.llm.model_capabilities import supports_prompt_caching

supports_prompt_caching("openai/gpt-4o")  # True
supports_prompt_caching("ollama/llama3")  # False

---

Configuration Options

Option	Type	Default	Description
enabled	bool	True	Response caching
prompt_caching	Optional[bool]	None	Provider prompt-cache hints (Anthropic manual, OpenAI automatic when eligible)

Set via Agent(caching=True) or Agent(caching=CachingConfig(...)).

The include_cache_boundary flag on build_cache_optimized_context() controls whether the returned dict contains the marker; it does not affect what is sent on the wire.

---

Best Practices

Use supported models

OpenAI, Anthropic, Bedrock, and Deepseek support caching — local models like Ollama do not.

Keep instructions stable

Changing system prompts between turns breaks the cached prefix.

Enable memory for full optimisation

Without memory, only tool sorting applies — memory=True activates the cache-optimised context path.

Let the SDK sort tools

Do not manually reorder tool lists — the SDK sorts deterministically by name.

---

Related

Prompt Caching

Deterministic memory ordering for cache hits

Prompt Caching CLI

Enable caching from the command line