Prompt Cache Optimization

Stable, cache-friendly prompts across memory, tools, and tasks for lower cost and latency

Turn on memory on a supported model — the SDK sorts tools and lays out context so providers can cache the stable prefix.

from praisonaiagents import Agent

agent = Agent(
    instructions="You are a helpful assistant",
    llm="openai/gpt-4o",
    memory=True,
)

agent.start("Summarise the latest report")
# Turn 2+ reuses cached system prompt + memory prefix
agent.start("What changed since yesterday?")

The user chats across turns; stable prefixes stay cached for lower cost.

How It Works

Quick Start

1. Simple Usage

   Automatic on supported models when memory is enabled:

   from praisonaiagents import Agent
   
   agent = Agent(
       instructions="You are a helpful assistant",
       llm="openai/gpt-4o",
       memory=True,
   )
   
   agent.start("Summarise the latest report")
   
2. With Configuration

   Make prompt caching explicit:

   from praisonaiagents import Agent, CachingConfig
   
   agent = Agent(
       instructions="You are a helpful assistant",
       llm="anthropic/claude-sonnet-4-20250514",
       memory=True,
       caching=CachingConfig(prompt_caching=True),
   )
   
   agent.start("Analyse the data trends")
   

---

How It Works

Behaviour	What it does
Deterministic tool order	Tools sorted by function name — reordering tools does not break cache
Stable memory prefix	Memory sections in fixed order via build_cache_optimized_context()
Reserved boundary constant	CACHE_BOUNDARY is defined for future structured-cache metadata — not injected into current prompts

CACHE_BOUNDARY is a reserved marker constant ("--- CACHE_BOUNDARY ---") on praisonaiagents.memory.memory for future structured-cache metadata. No current provider consumes it, so it is intentionally not appended to the outgoing system prompt. Providers hit the cache because the prefix is deterministic — not because of any marker.

On OpenAI and Gemini the SDK emits no explicit cache markers. Those providers apply prefix caching automatically as long as the request prefix is byte-identical across turns. The optimisations on this page — deterministic tool order, stable memory layout, no wall-clock in the system prompt — are what make automatic caching work.

Anthropic works differently: it gets explicit cache_control breakpoints on the system block and the stable history prefix. See Prompt Caching for the Anthropic breakpoint behaviour and the provider-behaviour table.

Check model support:

from praisonaiagents.llm.model_capabilities import supports_prompt_caching

supports_prompt_caching("openai/gpt-4o")  # True
supports_prompt_caching("ollama/llama3")  # False

---

Detecting Cache Invalidation

When a turn’s prefix changes and the provider cache misses, the gateway can tell you — via a hook event and a metric, automatically when metrics are on.

from praisonaiagents import Agent
from praisonaiagents.agent import prompt_prefix_signature

agent = Agent(
    instructions="You are a helpful assistant",
    llm="openai/gpt-4o",
    memory=True,
)

# Fingerprint whatever the provider will cache
sig_before = prompt_prefix_signature(agent)
# ... swap a tool or edit instructions ...
sig_after = prompt_prefix_signature(agent)
assert sig_before != sig_after  # cache miss next turn

Surface	Where	What it does
prompt_prefix_signature(agent)	from praisonaiagents.agent import prompt_prefix_signature	Returns a sha256 hexdigest of model + sorted tool-schema fingerprints + instructions
HookEvent.PROMPT_PREFIX_INVALIDATED	from praisonaiagents.hooks.types import HookEvent	Advisory hook fired when this turn’s signature differs from the last
prompt_cache_invalidations_total	Prometheus /metrics on the gateway	Counter incremented per drifted turn

Fingerprinted: model identity (from agent.llm, or .model, or str()), the sorted set of tool schema fingerprints (name + description + JSON-serialised params for dict tools; name + docstring for callables), and the agent.instructions string. Tool reordering keeps the signature stable.

Excluded (to avoid false positives): arrival time, routing facts, memory/RAG recall, and per-turn user text.

The PROMPT_PREFIX_INVALIDATED hook listener receives:

event_name = HookEvent.PROMPT_PREFIX_INVALIDATED
session_id = "<storage_key>"
agent_name = "<agent.agent_name or agent.name or 'bot'>"
timestamp  = "<time.time() as str>"
cwd        = "<os.getcwd()>"
extra      = {
    "old_sig": "<sha256 hex of previous turn's prefix>",
    "new_sig": "<sha256 hex of this turn's prefix>",
    "reason":  "tools/model",
}

Best-effort semantics: prompt_prefix_signature never raises — missing attributes degrade to empty strings. The baseline is cleared on session reap/clear/reset, so a reopened session does not fire a false invalidation. The metric only moves when GatewayMetrics is enabled.

---

Configuration Options

Option	Type	Default	Description
enabled	bool	True	Response caching
prompt_caching	Optional[bool]	None	Provider prompt-cache hints (Anthropic manual, OpenAI automatic when eligible)

Set via Agent(caching=True) or Agent(caching=CachingConfig(...)).

The include_cache_boundary flag on build_cache_optimized_context() controls whether the returned dict contains the marker; it does not affect what is sent on the wire.

---

Best Practices

Use supported models

OpenAI, Anthropic, Bedrock, and Deepseek support caching — local models like Ollama do not.

Keep instructions stable

Changing system prompts between turns breaks the cached prefix.

Enable memory for full optimisation

Without memory, only tool sorting applies — memory=True activates the cache-optimised context path.

Let the SDK sort tools

Do not manually reorder tool lists — the SDK sorts deterministically by name.

---

Related

Prompt Caching

Deterministic memory ordering for cache hits

Prompt Caching CLI

Enable caching from the command line