> ## Documentation Index
> Fetch the complete documentation index at: https://praison.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Offline Testing

> Unit-test agents without hitting a provider

Run agent tests offline — no API keys, no network, no bill — with `ScriptedModel` in place of a real provider and a hard gate that blocks any real request.

> [tab] Python
>
> ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
> from praisonaiagents import Agent, ScriptedModel
>
> model = ScriptedModel(["Paris."])
> agent = Agent(instructions="You are a geography bot.", llm=model)
>
> assert agent.start("What is the capital of France?") == "Paris."
> assert model.requests[0].last_user_message == "What is the capital of France?"
> ```
>
> `ScriptedModel` subclasses the real `LLM`, so everything around the call — system prompt assembly, tool schemas, the tool loop, streaming, async — runs production code; only the provider-facing methods are replaced.
>
> ```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
> graph LR
>     P[🧪 pytest] --> G{🚦 allow_model_requests<br/>False?}
>     G -->|blocked| X[🚫 ModelRequestBlocked<br/>names your call site]
>     G -->|scripted| S[🎬 ScriptedModel replies]
>     S --> A[🤖 Agent runs production path<br/>prompt · tools · retries · streaming]
>     A --> Assert[✅ assertions]
>
>     classDef test fill:#6366F1,stroke:#7C90A0,color:#fff
>     classDef gate fill:#F59E0B,stroke:#7C90A0,color:#fff
>     classDef bad fill:#8B0000,stroke:#7C90A0,color:#fff
>     classDef run fill:#189AB4,stroke:#7C90A0,color:#fff
>     classDef ok fill:#10B981,stroke:#7C90A0,color:#fff
>
>     class P test
>     class G gate
>     class X bad
>     class S,A run
>     class Assert ok
> ```

> [tab] TypeScript
>
> ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
> import { Agent, ScriptedModel, allowModelRequests } from 'praisonai';
>
> allowModelRequests(false);            // nothing in this suite may reach a provider
>
> const model = new ScriptedModel(['Refunded order A1.']);
> const agent = new Agent({ instructions: 'Support bot.', llm: model });
>
> expect(await agent.chat('refund order A1')).toBe('Refunded order A1.');
> expect(model.requestCount).toBe(1);
> expect(model.requests[0].prompt).toBe('refund order A1');
> ```
>
> `ScriptedModel` subclasses the real `BaseLLM`, replacing only `generate` / `generateStream` / `generateText` / `streamText` / `generateObject`. Prompt assembly, tool schemas, and the response shape all run production code — the double cannot drift from the real client because it *is* the real client with its network boundary swapped out.
>
> ```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
> graph LR
>     P[🧪 jest] --> G{🚦 allowModelRequests<br/>false?}
>     G -->|blocked| X[🚫 ModelRequestBlocked<br/>names your call site]
>     G -->|scripted| S[🎬 ScriptedModel replies]
>     S --> A[🤖 Agent runs production path<br/>prompt · tools · streaming]
>     A --> Assert[✅ expect(...)]
>
>     classDef test fill:#6366F1,stroke:#7C90A0,color:#fff
>     classDef gate fill:#F59E0B,stroke:#7C90A0,color:#fff
>     classDef bad fill:#8B0000,stroke:#7C90A0,color:#fff
>     classDef run fill:#189AB4,stroke:#7C90A0,color:#fff
>     classDef ok fill:#10B981,stroke:#7C90A0,color:#fff
>
>     class P test
>     class G gate
>     class X bad
>     class S,A run
>     class Assert ok
> ```
>
> > [!NOTE]
> >
> > **TypeScript differs from Python in a few ways:**
> >
> > * Method / helper naming is camelCase (`allowModelRequests`, `noModelRequests`, `modelRequestsAllowed`, `requestCount`) rather than snake\_case.
> > * `noModelRequests` takes a function (`await noModelRequests(async () => { ... })`); Python uses a `with` context manager.
> > * Scripting a tool call is **Python-only** for now — `ScriptedModel.tool_call(...)` does not yet exist in TS. Scripts are strings or `(req) => string` callables. Track parity on the [TS parity issue](https://github.com/MervinPraison/PraisonAI/pull/4964).
> > * The request gate uses a **counter** from day one, so an inner `noModelRequests()` scope exiting first cannot re-enable requests while an outer scope is still open.
> > * `ModelRequestBlocked` is a regular `Error` subclass. A `try { ... } catch (e) { ... }` around an agent call will catch it — rethrow, or check `e instanceof ModelRequestBlocked`.

## Quick Start

> [tab] Python
>
> **1. Install and gate the suite**
>
> Add the gate to `conftest.py` so nothing in the suite can reach a provider.
>
> ```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
> pip install praisonaiagents
> ```
>
> ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
> # conftest.py
> from praisonaiagents import allow_model_requests
>
> allow_model_requests(False)   # nothing in this suite may reach a provider
> ```
>
> **2. Script one text reply**
>
> ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
> from praisonaiagents import Agent, ScriptedModel
>
> model = ScriptedModel(["Paris."])
> agent = Agent(instructions="Geography bot.", llm=model)
>
> assert agent.start("Capital of France?") == "Paris."
> ```
>
> **3. Script a tool call + follow-up**
>
> ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
> from praisonaiagents import Agent, ScriptedModel
>
> def refund(order_id: str) -> str:
>     return f"refunded {order_id}"
>
> model = ScriptedModel([
>     ScriptedModel.tool_call("refund", {"order_id": "A1"}),
>     "Refunded order A1.",
> ])
> agent = Agent(instructions="Support bot.", llm=model, tools=[refund])
>
> assert agent.start("refund order A1") == "Refunded order A1."
> ```
>
> **4. Assert on what the agent sent**
>
> ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
> assert model.request_count == 2                 # tool turn, then the follow-up
> assert model.requests[0].tool_names == ("refund",)   # tools offered on turn 1
> assert model.requests[0].last_user_message == "refund order A1"
> ```

> [tab] TypeScript
>
> **1. Install and gate the suite**
>
> Add the gate to a Jest setup file so nothing in the suite can reach a provider.
>
> ```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
> npm install praisonai
> ```
>
> ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
> // jest.setup.ts
> import { allowModelRequests } from 'praisonai';
>
> allowModelRequests(false);   // nothing in this suite may reach a provider
> ```
>
> **2. Script one text reply**
>
> ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
> import { Agent, ScriptedModel } from 'praisonai';
>
> const model = new ScriptedModel(['Paris.']);
> const agent = new Agent({ instructions: 'Geography bot.', llm: model });
>
> expect(await agent.chat('Capital of France?')).toBe('Paris.');
> ```
>
> **3. Script a reply from the request**
>
> Script entries may be plain strings **or** functions of the recorded request (parity with Python's lambda).
>
> ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
> import { Agent, ScriptedModel } from 'praisonai';
>
> const model = new ScriptedModel([(req) => `You said: ${req.prompt}`]);
> const agent = new Agent({ instructions: 'Echo bot.', llm: model });
>
> expect(await agent.chat('hello')).toBe('You said: hello');
> ```
>
> **4. Assert on what the agent sent**
>
> ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
> expect(model.requestCount).toBe(1);
> expect(model.requests[0].prompt).toBe('hello');
> ```

## Canonical example

> [tab] Python
>
> ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
> # conftest.py
> from praisonaiagents import allow_model_requests
> allow_model_requests(False)          # nothing in this suite may reach a provider
> ```
>
> ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
> from praisonaiagents import Agent, ScriptedModel
>
> def refund(order_id: str) -> str:
>     return f"refunded {order_id}"
>
> model = ScriptedModel([
>     ScriptedModel.tool_call("refund", {"order_id": "A1"}),
>     "Refunded order A1.",
> ])
> agent = Agent(instructions="Support bot.", llm=model, tools=[refund])
>
> assert agent.start("refund order A1") == "Refunded order A1."
> assert model.request_count == 2
> assert model.requests[0].tool_names == ("refund",)
> ```

> [tab] TypeScript
>
> ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
> // jest.setup.ts
> import { allowModelRequests } from 'praisonai';
> allowModelRequests(false);           // nothing in this suite may reach a provider
> ```
>
> ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
> import { Agent, ScriptedModel } from 'praisonai';
>
> const model = new ScriptedModel(['Refunded order A1.']);
> const agent = new Agent({ instructions: 'Support bot.', llm: model });
>
> expect(await agent.chat('refund order A1')).toBe('Refunded order A1.');
> expect(model.requestCount).toBe(1);
> expect(model.requests[0].prompt).toBe('refund order A1');
> ```
>
> > [!NOTE]
> >
> > Scripting a tool call is **Python-only** today — there is no `ScriptedModel.toolCall(...)` in TS. Script entries are strings or `(req) => string` callables. A scripted model driving a retryable guardrail is a common test pattern — see [Retry preserves the conversation](https://praison.ai/docs/features/guardrails.md#retry-preserves-the-conversation).

## How it works

> [tab] Python
>
> `ScriptedModel` is a real `LLM` subclass that replaces only the methods which talk to a provider — nothing else.
>
> * System prompts are assembled normally, tools are serialised to real schemas, scripted tool calls are dispatched through the agent's own executor, and results are fed back as real tool messages.
> * Replies are built as genuine `litellm.ModelResponse` objects, parsed by the same code path a live response would take — so the double cannot drift from reality.
> * When the agent asks for one more reply than the script holds, the double raises `ScriptExhausted` (usually the script is one reply short — after a tool call the agent comes back for a follow-up answer).
>
> A script entry may be a `str` (a final answer), a `ScriptedModel.tool_call(...)`, a list of tool calls, a `ScriptedReply`, or a callable receiving the `RecordedRequest`:
>
> ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
> from praisonaiagents import ScriptedModel
>
> # A reply that depends on what the agent actually sent:
> model = ScriptedModel([lambda req: f"You said: {req.last_user_message}"])
> ```

> [tab] TypeScript
>
> `ScriptedModel` extends the real `BaseLLM` and replaces only `generate` / `generateStream` / `generateText` / `streamText` / `generateObject`. Everything else runs production code.
>
> * Prompt assembly, tool schemas, and the response shape all run the real client — the double cannot drift because it *is* the real client with its network boundary swapped out.
> * When the agent asks for one more reply than the script holds, the double throws `ScriptExhausted`.
>
> A script entry is a plain string (a final answer) or a callable receiving the `RecordedRequest`:
>
> ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
> import { ScriptedModel } from 'praisonai';
>
> // A reply that depends on what the agent actually sent:
> const model = new ScriptedModel([(req) => `You said: ${req.prompt}`]);
> ```

## The request gate

> [tab] Python
>
> `allow_model_requests(False)` blocks all six litellm request sites and both OpenAI client properties process-wide. Any attempt to reach a provider raises `ModelRequestBlocked`, whose message names the offending call site in **your** code.
>
> ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
> from praisonaiagents import Agent, allow_model_requests, ModelRequestBlocked
>
> allow_model_requests(False)
>
> agent = Agent(instructions="…", llm="gpt-4o")   # a real model, no script
> try:
>     agent.start("hello")
> except ModelRequestBlocked as e:
>     print(e.provider)    # e.g. "litellm.completion"
>     print(e.call_site)   # "<your_test>.py:42 in test_thing" — the line to fix
> ```
>
> Call `allow_model_requests(True)` to restore normal behaviour, or scope a single block with `no_model_requests()`:
>
> ```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
> from praisonaiagents import Agent, no_model_requests
>
> with no_model_requests():
>     Agent(instructions="…", llm="gpt-4o").start("hello")   # raises ModelRequestBlocked
> ```
>
> > [!WARNING]
> >
> > **`ModelRequestBlocked` and `ScriptExhausted` derive from `BaseException`, not `Exception` — on purpose.** The agent's tool loop catches `Exception` broadly and turns failures into a `None` answer. An ordinary exception raised from inside the double would reach your test as a mysterious `None`, hiding the very thing the test needs to be told ("your script is one reply short", "a real request leaked"). Deriving from `BaseException` lets these escape the loop — the same reasoning pytest uses for its own outcome exceptions. Do not wrap your agent calls in a bare `except Exception` that would swallow them.

> [tab] TypeScript
>
> `allowModelRequests(false)` turns off real provider calls for the whole process. Any attempt to reach a provider throws `ModelRequestBlocked`, whose message names the offending call site. `modelRequestsAllowed()` is a predicate you can check directly.
>
> Scope a single block with `noModelRequests()` — it takes a function and blocks real requests for its duration:
>
> ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
> import { Agent, noModelRequests } from 'praisonai';
>
> await noModelRequests(async () => {
>   const agent = new Agent({ instructions: '…', llm: 'gpt-4o' });
>   await agent.chat('hello');   // throws ModelRequestBlocked
> });
> ```
>
> > [!NOTE]
> >
> > The gate uses a **counter**, not a saved-and-restored boolean. An inner `noModelRequests()` scope exiting first cannot re-enable requests while an outer scope is still open — nested scopes compose safely.
>
> > [!WARNING]
> >
> > `ModelRequestBlocked` is a regular `Error` subclass, not a `BaseException`. The Agent's turn loop does not swallow it, but a `try { ... } catch (e) { ... }` block wrapping an agent call **will** catch it. Either rethrow, or check `e instanceof ModelRequestBlocked`.

## When to use offline vs. real-API testing

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
graph TB
    Start([What are you testing?]) --> Q{Wiring or behaviour?}
    Q -->|Tool routing, retries,<br/>guardrails, streaming| Off[🧪 Offline<br/>ScriptedModel]
    Q -->|Prompt quality, model<br/>behaviour, provider cost| Real[🔑 Real API<br/>RUN_REAL_KEY_TESTS]

    classDef start fill:#6366F1,stroke:#7C90A0,color:#fff
    classDef question fill:#F59E0B,stroke:#7C90A0,color:#fff
    classDef off fill:#10B981,stroke:#7C90A0,color:#fff
    classDef real fill:#189AB4,stroke:#7C90A0,color:#fff

    class Start start
    class Q question
    class Off off
    class Real real
```

Offline tests assert what the agent *does* — which tool it calls, in what order, with what arguments. For prompt quality and model behaviour, use [Real API Testing](https://praison.ai/docs/features/real-api-testing.md).

### Testing a bot turn offline

Pair `ScriptedModel` with `LocalBot`'s injected `stdin`/`stdout` to assert a full hook → session → agent → send turn with no API key, no network, and no live channel token. See [Local Channel → Testing / CI](https://praison.ai/docs/features/local-channel.md#testing--ci--offline-harness).

## `ScriptedModel` configuration

> [tab] Python
>
> | Option | Type | Default | Description |
> | - | - | - | - |
> | `script` | `Iterable[str \| ScriptedReply \| ScriptedToolCall \| dict \| callable]` | `()` | Ordered replies. Strings are text answers; use `ScriptedModel.tool_call(...)` for a tool call; a callable receives the `RecordedRequest`. |
> | `model` | `str` (keyword-only) | `"scripted/model"` | Fake model id reported by the agent. Set a real id (e.g. `"gpt-4o"`) to exercise model-specific behaviour while still answering from the script. |
> | `**llm_kwargs` | — | — | Forwarded to the underlying `LLM`. |
>
> Inspection surface after a run:
>
> | Attribute / method | Returns | Use |
> | - | - | - |
> | `model.request_count` | `int` | How many turns the agent sent. |
> | `model.requests` | `tuple[RecordedRequest, …]` | Every request, in order. |
> | `requests[i].tool_names` | `tuple[str, …]` | Tool schemas offered on that turn. |
> | `requests[i].last_user_message` | `str \| None` | Most recent user message text. |
> | `requests[i].system_prompt` | `str \| None` | The assembled system prompt. |
> | `requests[i].params` | `dict` | Every completion param sent (`tool_choice`, `response_format`, …). |
> | `requests[i].tool_results` | `tuple[dict, …]` | Tool-result messages in that request's history. |
>
> `ScriptedReply`, `ScriptedToolCall`, and `RecordedRequest` are importable from `praisonaiagents.model_harness` when you need to build or type them explicitly.

> [tab] TypeScript
>
> The constructor takes an ordered script and an optional config:
>
> ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
> new ScriptedModel(
>   script: Array<string | ((request: RecordedRequest) => string)>,
>   config?: Partial<LLMConfig>,   // model defaults to "scripted/test-model"
> )
> ```
>
> Inspection surface after a run:
>
> | Attribute / method | Returns | Use |
> | - | - | - |
> | `model.requestCount` | `number` | How many completions the agent asked for. |
> | `model.requests` | `RecordedRequest[]` | Every request the agent sent, in order. |
> | `model.requests[i].prompt` | `string` | The last user-message text on turn *i*. |
> | `model.requests[i].systemPrompt` | `string \| undefined` | The assembled system prompt. |
> | `model.requests[i].options` | `LLMGenerateOptions \| undefined` | The full options object passed to `generate`. |
> | `model.remaining` | `number` | Replies left unconsumed. |
> | `model.reset()` | `void` | Rewind so the same instance can drive another run. |
> | `model.extend(...entries)` | `void` | Append more replies to the script. |

## Best Practices

> [accordion] Gate the whole session in a fixture
>
> Put `allow_model_requests(False)` in a session-scoped fixture (or `conftest.py`) so a forgotten mock cannot leak a real call and quietly bill you.

> [accordion] Script the arguments the agent will really produce
>
> Assert `requests[i]` reflects the call you expect — a scripted `tool_call("refund", {"order_id": "A1"})` should mirror what your instructions steer the model toward.

> [accordion] Prefer real ModelResponse shapes
>
> Script with strings and `ScriptedModel.tool_call(...)` rather than inventing raw provider dicts — the helpers build genuine `litellm` response objects that the production parser understands.

> [accordion] Never mix scripted and real in one session
>
> Keep offline tests and `RUN_REAL_KEY_TESTS` integration tests in separate sessions. `allow_model_requests(False)` blocks the whole process, so a real-API test in the same run would fail with `ModelRequestBlocked`.

## Related

> [card] [Real API Testing](https://praison.ai/docs/features/real-api-testing.md)
>
> Gated integration tests against live providers.

> [card] [Guardrails](https://praison.ai/docs/features/guardrails.md)
>
> Test guardrail wiring offline with scripted replies.

> [card] [Tools](https://praison.ai/docs/features/tools.md)
>
> How the tool loop the double exercises works.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.