Persistence Overview

Database persistence for PraisonAI Agents

Persistence Overview

PraisonAI supports automatic database persistence for conversations, knowledge, and state management across 22 database backends.

Quick Start

Enable persistence in 2 lines:

from praisonaiagents import Agent

agent = Agent(
    name="Assistant",
    memory={
        "db": "postgresql://localhost/mydb",
        "session_id": "my-session"
    }
)
response = agent.start("Hello!")  # Auto-persists
print(response)

Installation

pip install "praisonaiagents[tools]"

Supported Backends

Category	Backends	Count
Conversation	PostgreSQL, MySQL (including MySQLConversationStore), SQLite, SingleStore, Supabase, SurrealDB, Turso	7+
Knowledge	Qdrant, ChromaDB, Pinecone, Weaviate, LanceDB, Milvus, PGVector, Redis, Cassandra, ClickHouse, MongoDB Vector, Couchbase, SingleStore Vector, SurrealDB Vector, Upstash Vector, LightRAG, LangChain, LlamaIndex, CosmosDB	19
State	Redis, MongoDB, DynamoDB, Firestore, Upstash, Memory, GCS	7

Backend Aliases

The persistence registry provides user-friendly aliases for common databases:

Conversation stores:

• neon, cockroachdb, xata → postgres
• asyncpg, postgres_async → async_postgres
• aiomysql, mysql_async → async_mysql
• sqlite_sync → sync_sqlite
• aiosqlite, sqlite_async → async_sqlite
• libsql → turso

Knowledge stores:

• chromadb → chroma
• mongodb_atlas, cosmos, azure_cosmos → Vector store variants
• llama_index, langchain_adapter → Framework adapters

State stores:

• motor, mongodb_async → async_mongodb (fully implements the StateStore contract as of PR #4287 — hash / TTL / keys operations are now available on the async backend). See Async MongoDB (motor).

Non-blocking prefix scan (Redis, Upstash)

PraisonAIDB.get_runs() and get_traces() iterate keys via a cursor-based SCAN on Redis / Upstash rather than the blocking KEYS command. Other backends implement scan_prefix() as keys(f"{prefix}*"), so nothing ever falls back to an unbounded KEYS *.

Reference: PraisonAI PR #3812. The StateStore.scan_prefix(prefix) method is backend plumbing — self-hosters on Redis / Upstash get a “recent runs” read that never stalls other clients on the same instance. See Run History.

Architecture

┌─────────────────────────────────────────┐
│           praisonaiagents.Agent         │
│         (memory={db: "..."})            │
└─────────────────┬───────────────────────┘
                  │
┌─────────────────▼───────────────────────┐
│     Memory/Knowledge/State Adapters     │
│         (DbAdapter Protocol)            │
└─────────────────┬───────────────────────┘
                  │
┌─────────────────▼───────────────────────┐
│        Database Backends Layer          │
├─────────────┬─────────────┬─────────────┤
│ Conversation│  Knowledge  │    State    │
│   Store     │    Store    │    Store    │
└─────────────┴─────────────┴─────────────┘

Two Entry Points, One Owner

PraisonAIDB is the single owner of the “agent lifecycle → store” write hooks; PersistenceOrchestrator is a thin façade that delegates those hooks to it.

Both entry points share the same store-write logic:

• PraisonAIDB (the live path) owns the lifecycle-hook and store-dispatch logic — on_message / aon_message, on_agent_end / aon_agent_end, and the sync/async dispatch decision. MemoryConfig(db=PraisonAIDB(...)) routes straight to it and is the recommended user-facing wiring.
• PersistenceOrchestrator delegates its store-write side to a PraisonAIDB instance built lazily from its own stores (internal PraisonAIDB._from_stores(...) plumbing), while keeping its unique surface: the resume-aware on_agent_start return contract, the LRU session cache, and the knowledge / state / context helpers.
• The orchestrator path (via wrap_agent_with_persistence, PersistentAgent, or create_persistent_session) continues to work unchanged and now shares the same store-write logic under the hood.

Reference: PraisonAI PR #4394 (fixes #4383). The public API is unchanged — this is an internal consolidation. _from_stores is internal plumbing, not a public API.

Key Features

• Zero Config: SQLite works out of the box with memory=True
• Session Resume: praisonai session resume <id> is a first-class restore — it brings back chat history, model, and agent name, not just a transcript. Pass a follow-up prompt to continue immediately: praisonai session resume <id> "<prompt>".
• Runtime State Mirroring: lightweight per-turn artefacts for native↔plugin handoff (opt-in via SessionConfig.mirror_runtime_state)
• Lazy Loading: No performance impact until used
• CLI Support: praisonai persistence doctor/run/resume

Concurrency & Thread Safety

Persistence sync wrappers (get_session, add_message, get, set, delete, list_keys, clear, close, …) are safe to call from any context — plain sync scripts, worker threads, or code running inside a live event loop (FastAPI, Streamlit, background tasks). They route through a canonical async bridge, so you will not see RuntimeError: This event loop is already running anymore.

Async hooks offloading (PR #1829): Sync conversation stores plugged into async hooks are now automatically offloaded via asyncio.to_thread() rather than blocking the event loop. PraisonAIDB owns this dispatch (_call_store / _dispatch_async): it uses isinstance(store, AsyncConversationStore) to decide whether to await the store directly or wrap it in thread execution. As of PR #4394, PersistenceOrchestrator delegates its store writes through these same helpers rather than keeping a second copy, so the offloading behaviour is identical no matter which entry point you use.

Async state-store hook naming (PR #4287): The DB adapter’s async lifecycle hooks now call async_set / async_get on the state store rather than the non-existent aset / aget names. Previously the async path silently fell back to sync methods off-loaded to a thread that re-entered the async bridge, pinning the event loop under load. No user action — async agents backed by any state store (Redis, MongoDB, DynamoDB, …) now stay non-blocking end-to-end.

Async RAG offload (PR #3837): PersistenceOrchestrator.aretrieve_knowledge() / aadd_knowledge() await native async knowledge stores and offload sync stores via asyncio.to_thread, so RAG calls from an async agent no longer block the event loop. See Async Knowledge Retrieval.

Non-lossy tool-call persistence (PR #3837)

Tool call results are persisted verbatim; the 1000-character truncation is gone.

PraisonAIDB.on_tool_call / aon_tool_call previously wrote str(result)[:1000], silently dropping any tool history longer than 1 KB. Both paths now delegate to a shared _serialize_tool_call(tool_name, args, result) helper that stores the full result. An agent that resumes a session sees the complete tool output it produced, so downstream reasoning that depended on the tail of a long tool result now works after resume.

No user action required — the behaviour is on by default, and existing rows written with truncation are unaffected (only new writes change). Reference: PraisonAI PR #3837.

As of PR #1763, you can also opt into the dedicated sync_sqlite backend (mode="sync") — it provides per-call connection locking (threading.RLock) for multi-agent scenarios where you’d rather not depend on the legacy sync wrappers.

The DbAdapter’s lazy store initialisation is also race-free: the first thread to touch the adapter constructs the stores, and subsequent threads see the ready instance. Since PR #4394, PersistenceOrchestrator’s message-write and session-end operations delegate to PraisonAIDB (through _call_store / _dispatch_async), so this race-free guarantee now covers both entry points.

The session cache inside PersistenceOrchestrator is also thread-safe as of PR #1609. Reads return a deepcopy of the cached ConversationSession, so multiple agents sharing one orchestrator can read/update sessions concurrently without races.

The JSON session store (DefaultSessionStore) reloads from disk under FileLock for every mutator as of PR #1709 (metadata) and PR #1724 (agent info, gateway info, clear). Two processes pointed at the same ~/.praisonai/sessions/ directory can now interleave add_message and set_agent_info / clear_session / set_gateway_info calls without losing messages. Reads reload from disk under lock on every get_chat_history / get_session call.

HierarchicalSessionStore inherits the JSON store’s reload-under-lock guarantees and additionally preserves parent_id, children_ids, and snapshots across all mutators (PR #1745). UI hosts that fork sessions or take snapshots are safe to run alongside an agent’s auto_save writer.

SqliteTranscriptStore (PraisonAI PR #3409) subclasses DefaultSessionStore but swaps the FileLock-guarded JSON files for one WAL SQLite row per session, wrapping each read-modify-write in a BEGIN IMMEDIATE transaction for cross-process append-safety. It is the default gateway transcript backend when session.persist: true — which is now the default (see Gateway Session Persistence and SQLite Transcript Store).

SQLite conversation store — clean shutdown across threads

SQLiteConversationStore opens one connection per calling thread (threading.local). As of PraisonAI #4537, store.close() drains every tracked connection, not just the caller’s — safe to call from a FastAPI lifespan hook, a shutdown supervisor, or atexit after fanning the store across a worker pool. Long-lived deployments (praisonai serve, gateway, async job pool, kanban) no longer leak an fd + SQLite memory arena per worker thread.

Under check_same_thread=True, connections owned by another thread can only be closed by that thread; those stay tracked (not silently dropped) so the owning thread’s later close() finishes the drain. No sqlite3.ProgrammingError reaches user code.

These improvements were added in PraisonAI PR #1466, #1609, #1709, #1724, and #1727 to ensure robust multi-threaded operation in production environments.

The JSON session store (DefaultSessionStore) reloads from disk under FileLock for every mutator as of PR #1709 (metadata), PR #1724 (agent info, gateway info, clear), and PR #1727 (LocalManagedAgent._persist_state). PRs #1759 and #1764 extended the same FileLock-guarded reload to the read path (get_chat_history, get_session, get_sessions_by_agent), so two processes pointed at the same ~/.praisonai/sessions/ directory observe each other’s writes without any stale-cache window.

Session Cache Size

PersistenceOrchestrator keeps recently-touched sessions in a bounded in-memory LRU (OrderedDict) so long-running servers/bots don’t grow forever when every request carries a fresh session_id.

Env Var	Default	Description
PRAISONAI_SESSION_CACHE_MAX	1024	Max cached sessions (minimum 1). Set higher for high-cardinality session workloads, lower for memory-constrained hosts. Empty/non-numeric values fall back to the default (no crash).

# Raise the cap for a chat server that juggles many concurrent sessions
export PRAISONAI_SESSION_CACHE_MAX=8192

Reference: PraisonAI PR #3792. Eviction is least-recently-used — reads and writes mark recency, and the oldest session is dropped once the cap is reached.

Schema Versioning (PR #1829)

New unified schema system: All SQL conversation stores now inherit from _SQLConversationStoreBase with consistent SCHEMA_VERSION = "1.0.0". This applies to PostgreSQL, MySQL (new), and all existing SQL backends uniformly.

The base class provides:

• Standardized table schemas for sessions and messages
• Dialect-specific type mappings (_id_type, _json_type, _float_type)
• Unified retry logic with max_retries and retry_delay parameters
• Automatic serverless database detection and exponential backoff
• Consistent table_prefix handling across all SQL stores

MySQL backend: The new MySQLConversationStore (from praisonai.persistence.conversation.mysql_new) includes PlanetScale auto-retry with exponential backoff for serverless cold-starts.

Schema Migration (PR #1597)

Breaking change: All async conversation stores (async_sqlite, async_postgres, async_mysql) now default to table_prefix="praison_" (previously "praisonai_"). Sync stores were already on praison_.

If you ran an async store before this release, either:

1. Pass table_prefix="praisonai_" explicitly to keep your old tables, or
2. Rename existing tables: ALTER TABLE praisonai_sessions RENAME TO praison_sessions; (and likewise for _messages).

New columns: Async session/message schemas now include state (sessions), and tool_calls + tool_call_id (messages). The store creates them automatically on first connect via CREATE TABLE IF NOT EXISTS. Existing tables on a pre-#1597 schema will not auto-migrate; add the columns with:

ALTER TABLE praison_sessions ADD COLUMN state TEXT;
ALTER TABLE praison_messages ADD COLUMN tool_calls TEXT;
ALTER TABLE praison_messages ADD COLUMN tool_call_id TEXT;

The default (zero-config) JSON session store now carries the same tool_calls / tool_call_id fields automatically as of PraisonAI PR #3099 — no migration needed. The file format stays backward compatible: old text-only session files load unchanged, and the new fields are only written when a message actually has them. See Session Resume.

Custom Stores & Aliases

Register custom storage backends with the persistence registry, including optional aliases:

from praisonai.persistence import get_default_registry
from praisonaiagents import Agent

def my_custom_store(connection_string, **kwargs):
    return CustomStoreImplementation(connection_string, **kwargs)

# Register on the shared default so Agent(memory={"db": "my_store://..."}) sees it
registry = get_default_registry("conversation")
registry.register_store(
    "my_store",
    my_custom_store,
    aliases=("mine", "ms", "custom_db"),
)

# All of these now resolve to my_custom_store
agent = Agent(memory={"db": "my_store://config"})
agent = Agent(memory={"db": "mine://config"})       # alias
agent = Agent(memory={"db": "ms://config"})         # alias
agent = Agent(memory={"db": "custom_db://config"})  # alias

The shared default registry is what agents use at runtime. Constructing a fresh StoreRegistry(...) registers backends only on that isolated instance — use get_default_registry("conversation") when you want Agent(memory=...) to see your backend immediately.

The register_store() method is the canonical API. The register() method is preserved as a backward-compatible alias.

Next Steps

• Quickstart - Get started in 5 minutes
• Session Resume - Continue conversations
• Session Runtime State - Per-turn runtime artefacts for replay and handoff
• Async Conversation Store - AsyncConversationStore protocol for non-blocking persistence
• CLI Reference - Command-line usage
• Backend Plugins - Custom storage backends