> ## Documentation Index
> Fetch the complete documentation index at: https://praison.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Observability

> Trace your Agents with Langfuse and LangSmith delivery plus 12 in-memory-only adapters

PraisonAI TypeScript records every LLM call, tool execution, and Agent decision as traces. **Langfuse** (install `langfuse`) and **LangSmith** (set `LANGSMITH_API_KEY`) deliver those traces to a real backend. The other 12 external adapters are wired in but **record to memory only** — they do not send traces to their vendor backends today. The built-in `console` and `memory` adapters record locally.

> [!WARNING]
>
> Only **Langfuse** (with the `langfuse` npm package installed) **and LangSmith** (with `LANGSMITH_API_KEY` set) ship traces to an external vendor. `console` and `memory` record locally. The other 12 external adapters accept traces and hold them in memory only — no network call is made, and setting the vendor's env var does not enable delivery. See the [Delivery Matrix](#observability-delivery-matrix) below.

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
graph LR
    Agent[Agent] --> Obs[Observability]
    Obs --> Trace([Trace])

    classDef agent fill:#8B0000,stroke:#7C90A0,color:#fff
    classDef tool fill:#189AB4,stroke:#7C90A0,color:#fff

    class Agent agent
    class Obs,Trace tool
    classDef agent fill:#8B0000,color:#fff
    classDef tool fill:#189AB4,color:#fff

```

## Observability Delivery Matrix

One table decides where your traces actually go today. This mirrors the `delivers` field in the SDK (`src/observability/types.ts`).

| Adapter | Delivers to vendor today? | Backend used |
| - | - | - |
| `langfuse` | ✅ (requires `npm install langfuse`) | Langfuse Cloud / self-hosted |
| `langsmith` | ✅ (requires `LANGSMITH_API_KEY`) | LangSmith Cloud / self-hosted |
| `console` | ❌ (records locally to stdout) | stdout |
| `memory` | ❌ (records locally in-process) | in-process array (read via `getAllTraces()`) |
| `noop` | ❌ (records nothing) | nothing |
| `arize`, `axiom`, `braintrust`, `helicone`, `laminar`, `langwatch`, `maxim`, `patronus`, `scorecard`, `signoz`, `traceloop`, `weave` | ❌ *(in-memory only)* | none — traces stay in the adapter's memory buffer |

> [!NOTE]
>
> `console` and `memory` are genuine local recorders: they do exactly what they claim (write to stdout / hold spans in memory), but they do not deliver to an external vendor. The 12 external adapters above record to memory and discard on exit.

## Supported Observability Tools

The `Env Variable` column shows what the SDK reads. **Setting the variable does not enable delivery** for any adapter marked `❌` — those traces stay in memory. The CLI marks these with `[no delivery - in-memory only]`.

| Tool | Description | Env Variable | Delivers to vendor? |
| - | - | - | - |
| **Langfuse** | Langfuse observability platform | `LANGFUSE_SECRET_KEY` | ✅ (with `npm install langfuse`) |
| **LangSmith** | LangSmith by LangChain | `LANGSMITH_API_KEY` *(or legacy `LANGCHAIN_API_KEY`)* | ✅ (requires `LANGSMITH_API_KEY`) |
| **LangWatch** | LangWatch monitoring | `LANGWATCH_API_KEY` | ❌ (in-memory only) |
| **Arize AX** | Arize AX (Phoenix) | `ARIZE_API_KEY` | ❌ (in-memory only) |
| **Axiom** | Axiom logging and analytics | `AXIOM_TOKEN` | ❌ (in-memory only) |
| **Braintrust** | Braintrust AI evaluation | `BRAINTRUST_API_KEY` | ❌ (in-memory only) |
| **Helicone** | Helicone observability proxy | `HELICONE_API_KEY` | ❌ (in-memory only) |
| **Laminar** | Laminar AI observability | `LMNR_PROJECT_API_KEY` | ❌ (in-memory only) |
| **Maxim** | Maxim AI testing | `MAXIM_API_KEY` | ❌ (in-memory only) |
| **Patronus** | Patronus AI evaluation | `PATRONUS_API_KEY` | ❌ (in-memory only) |
| **Scorecard** | Scorecard AI testing | `SCORECARD_API_KEY` | ❌ (in-memory only) |
| **SigNoz** | SigNoz OpenTelemetry | `SIGNOZ_ACCESS_TOKEN` | ❌ (in-memory only) |
| **Traceloop** | Traceloop OpenLLMetry | `TRACELOOP_API_KEY` | ❌ (in-memory only) |
| **Weave** | Weights & Biases Weave | `WANDB_API_KEY` | ❌ (in-memory only) |
| **Console** | Console logging (built-in) | (none) | ❌ (local stdout) |
| **Memory** | In-memory storage (built-in) | (none) | ❌ (local in-process) |

## Quick Start

**1. Simple Usage**

```typescript theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
import { Agent, createObservabilityAdapter, setObservabilityAdapter } from 'praisonai';

// Create adapter (langfuse, langsmith, console, memory, etc.)
const obs = await createObservabilityAdapter('langfuse');
setObservabilityAdapter(obs);

const agent = new Agent({
  name: 'Support Agent',
  instructions: 'Help users with their questions.'
});

// All Agent actions are now traced
await agent.chat('How do I reset my password?');

// Flush traces to the backend
await obs.flush();
```

**2. With Configuration**

```typescript theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
import { Agent, MemoryObservabilityAdapter, setObservabilityAdapter } from 'praisonai';

const obs = new MemoryObservabilityAdapter();
setObservabilityAdapter(obs);

const agent = new Agent({
  name: 'Support Agent',
  instructions: 'Help users with their questions.',
  observability: true
});

await agent.chat('How do I reset my password?');
```

## Agent with Tracing

```typescript theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
import { Agent, MemoryObservabilityAdapter, setObservabilityAdapter } from 'praisonai';

// Enable observability globally
const obs = new MemoryObservabilityAdapter();
setObservabilityAdapter(obs);

const agent = new Agent({
  name: 'Support Agent',
  instructions: 'Help users with their questions.',
  observability: true  // Enable tracing for this Agent
});

// All Agent actions are now traced
const response = await agent.chat('How do I reset my password?');

// View what happened
const traces = obs.getAllTraces();
for (const trace of traces) {
  console.log(`Trace: ${trace.name}`);
  for (const span of trace.spans) {
    console.log(`  ${span.type}: ${span.name} (${span.duration}ms)`);
  }
}
```

## Multi-Agent Tracing

Track interactions between multiple Agents:

```typescript theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
import { Agent, Agents, MemoryObservabilityAdapter, setObservabilityAdapter } from 'praisonai';

const obs = new MemoryObservabilityAdapter();
setObservabilityAdapter(obs);

const researcher = new Agent({
  name: 'Researcher',
  instructions: 'Research topics thoroughly.',
  observability: true
});

const writer = new Agent({
  name: 'Writer',
  instructions: 'Write clear summaries.',
  observability: true
});

const agents = new AgentTeam({
  agents: [researcher, writer],
  tasks: [
    { agent: researcher, description: 'Research: {topic}' },
    { agent: writer, description: 'Summarize the research' }
  ],
  observability: true  // Trace the entire workflow
});

await agents.start({ topic: 'AI trends 2024' });

// Analyze the workflow
const traces = obs.getAllTraces();
console.log(`Total traces: ${traces.length}`);
console.log(`Total LLM calls: ${traces.flatMap(t => t.spans).filter(s => s.type === 'llm').length}`);
```

## Agent Performance Monitoring

Track latency, tokens, and costs:

```typescript theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
import { Agent, createTool, MemoryObservabilityAdapter, setObservabilityAdapter } from 'praisonai';

const obs = new MemoryObservabilityAdapter();
setObservabilityAdapter(obs);

const agent = new Agent({
  name: 'Analytics Agent',
  instructions: 'Analyze data and provide insights.',
  observability: true
});

// Run multiple queries
await agent.chat('Analyze sales trends');
await agent.chat('Compare Q1 vs Q2');
await agent.chat('Predict Q3 performance');

// Analyze Agent performance
const traces = obs.getAllTraces();
const llmSpans = traces.flatMap(t => t.spans).filter(s => s.type === 'llm');

const stats = {
  totalCalls: llmSpans.length,
  avgLatency: llmSpans.reduce((sum, s) => sum + s.duration, 0) / llmSpans.length,
  totalTokens: llmSpans.reduce((sum, s) => sum + (s.attributes?.tokens || 0), 0)
};

console.log('Agent Performance:');
console.log(`  LLM Calls: ${stats.totalCalls}`);
console.log(`  Avg Latency: ${stats.avgLatency.toFixed(0)}ms`);
console.log(`  Total Tokens: ${stats.totalTokens}`);
```

## Agent Debugging

Debug Agent decisions and tool calls:

```typescript theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
import { Agent, createTool, ConsoleObservabilityAdapter, setObservabilityAdapter } from 'praisonai';

// Console adapter logs everything in real-time
const obs = new ConsoleObservabilityAdapter();
setObservabilityAdapter(obs);

const searchTool = createTool({
  name: 'search',
  description: 'Search for information',
  parameters: {
    type: 'object',
    properties: { query: { type: 'string' } },
    required: ['query']
  },
  execute: async ({ query }) => `Results for: ${query}`
});

const agent = new Agent({
  name: 'Debug Agent',
  instructions: 'Search for information to answer questions.',
  tools: [searchTool],
  observability: true
});

await agent.chat('Find information about TypeScript');

// Console output shows:
// [TRACE START] agent-execution
//   [SPAN START] llm-call (llm)
//   [SPAN END] llm-call - completed (1234ms)
//   [SPAN START] tool-call (tool) - search
//   [SPAN END] tool-call - completed (56ms)
//   [SPAN START] llm-call (llm)
//   [SPAN END] llm-call - completed (987ms)
// [TRACE END] agent-execution - completed
```

## Langfuse Integration (the delivery recipe)

Langfuse is the one external adapter that ships traces to a real backend. `langfuse` is an **optional peer dependency** of `praisonai` \u2014 install it separately:

```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
npm install langfuse
```

`isEnabled` becomes `true` only after the adapter successfully creates a Langfuse client from that package. If the package is missing, the adapter falls back to memory and `flush()` warns that nothing was delivered.

```typescript theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
import { Agent, createObservabilityAdapter, setObservabilityAdapter } from 'praisonai';

const langfuse = await createObservabilityAdapter('langfuse', {
  name: 'langfuse',
  apiKey: process.env.LANGFUSE_SECRET_KEY,
  baseUrl: process.env.LANGFUSE_HOST
});

console.log(langfuse.isEnabled); // true once the langfuse client is created

setObservabilityAdapter(langfuse);

const agent = new Agent({
  name: 'Production Agent',
  instructions: 'Handle customer inquiries.',
  observability: true
});

// All traces automatically sent to Langfuse
await agent.chat('I need help with my order');

// View in Langfuse dashboard:
// - Trace timeline
// - Token usage
// - Latency breakdown
// - Cost tracking
```

## LangSmith Integration (the delivery recipe)

LangSmith delivers traces to its REST Runs API using the `axios` dependency already shipped with `praisonai` — no extra package needed. `isEnabled` becomes `true` once `initialize()` finds `LANGSMITH_API_KEY` and builds the HTTP client.

```typescript theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
import { Agent, createObservabilityAdapter, setObservabilityAdapter } from 'praisonai';

const obs = await createObservabilityAdapter('langsmith');  // reads LANGSMITH_API_KEY
setObservabilityAdapter(obs);

const agent = new Agent({
  name: 'Support Agent',
  instructions: 'Help users with their questions.',
  observability: true
});

await agent.chat('How do I reset my password?');
await obs.flush();
// Open https://smith.langchain.com → your project → the trace is live
```

Self-hosted LangSmith works by setting `LANGSMITH_ENDPOINT`. The endpoint must be `http(s)`; plaintext `http://` is only accepted for loopback hosts (`localhost`, `127.0.0.1`, `::1`) so the API key never travels in the clear.

## How to know if my traces are actually going anywhere

Check `isEnabled`. The 12 non-delivering external adapters have no transport, so it is always `false` \u2014 this is not a config bug.

```typescript theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
import { createObservabilityAdapter } from 'praisonai';

const obs = await createObservabilityAdapter('weave');
console.log(obs.isEnabled);
// false \u2014 the weave adapter has no transport yet; traces stay in memory.

const langfuse = await createObservabilityAdapter('langfuse');
console.log(langfuse.isEnabled);
// true once the langfuse client is created (langfuse package installed + keys set).

const langsmith = await createObservabilityAdapter('langsmith');
console.log(langsmith.isEnabled);
// true once the HTTP client is built (LANGSMITH_API_KEY set + valid endpoint).
```

Constructing any of the 12 non-delivering adapters prints a `console.warn`, and calling `flush()` on them warns that nothing was delivered:

```
[OBSERVABILITY] The "weave" adapter does not send traces to weave. Delivery is not
implemented: traces are held in memory only and are lost when the process exits.
isEnabled is false for this reason. Use the "langfuse" adapter, or the built-in
"console"/"memory" adapters, if you need trace output.
```

## Why an in-memory adapter for a vendor I've never used?

The 12 non-delivering adapters still record spans to an in-memory buffer you can drain via `getAllTraces()` or route through a custom `flush()`. They share the `UndeliveredObservabilityAdapter` base class (`src/observability/adapters/external/undelivered.ts`) and act as placeholders for future real delivery. Until then, use `langfuse` or `langsmith` for vendor delivery or `console`/`memory` for local development.

## Custom Observability for Agents

Build custom monitoring:

```typescript theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
import { Agent, ObservabilityAdapter, TraceContext, SpanContext } from 'praisonai';

class DatadogAgentAdapter implements ObservabilityAdapter {
  startTrace(name: string, metadata?: Record<string, any>): TraceContext {
    // Send to Datadog APM
    const traceId = datadogTracer.startSpan(name, { tags: metadata });
    return {
      traceId,
      name,
      startSpan: (spanName, type) => this.createSpan(traceId, spanName, type),
      end: (status) => datadogTracer.finishSpan(traceId, { status })
    };
  }
  
  private createSpan(traceId: string, name: string, type: string): SpanContext {
    const spanId = datadogTracer.startChildSpan(traceId, name, { type });
    return {
      spanId,
      name,
      type,
      setAttributes: (attrs) => datadogTracer.setTags(spanId, attrs),
      addEvent: (event, data) => datadogTracer.addEvent(spanId, event, data),
      end: (status) => datadogTracer.finishSpan(spanId, { status })
    };
  }
}

// Use with Agents
setObservabilityAdapter(new DatadogAgentAdapter());
```

## Agent Audit Logging

Track Agent actions for compliance:

```typescript theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
import { Agent, createTool, MemoryObservabilityAdapter, setObservabilityAdapter } from 'praisonai';

const obs = new MemoryObservabilityAdapter();
setObservabilityAdapter(obs);

const sensitiveDataTool = createTool({
  name: 'access_customer_data',
  description: 'Access customer information',
  parameters: {
    type: 'object',
    properties: { customerId: { type: 'string' } },
    required: ['customerId']
  },
  execute: async ({ customerId }) => {
    // Tool execution is automatically traced
    return `Customer data for ${customerId}`;
  }
});

const agent = new Agent({
  name: 'Support Agent',
  instructions: 'Help customers with their accounts.',
  tools: [sensitiveDataTool],
  observability: true
});

await agent.chat('Look up customer C-12345');

// Generate audit log
const auditLog = obs.getAllTraces().flatMap(t => 
  t.spans.filter(s => s.type === 'tool').map(s => ({
    timestamp: s.startTime,
    action: s.name,
    agent: t.metadata?.agentName,
    details: s.attributes
  }))
);

console.log('Audit Log:', JSON.stringify(auditLog, null, 2));
```

## Switching Observability Tools

> [!WARNING]
>
> Switching adapters is not equivalent. Only `langfuse`, `langsmith`, `console`, and `memory` produce user-visible artifacts today (`noop` records nothing). The other 12 adapters accept traces and discard them on exit \u2014 switching to `weave` or `arize` records to memory only, no matter what env var you set.

```typescript theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
import { createObservabilityAdapter, setObservabilityAdapter } from 'praisonai';

// Switch between tools easily
const tool = process.env.OBS_TOOL || 'memory';
const obs = await createObservabilityAdapter(tool as any);
setObservabilityAdapter(obs);

// Supported tools: langfuse, langsmith, langwatch, arize, axiom,
// braintrust, helicone, laminar, maxim, patronus, scorecard,
// signoz, traceloop, weave, console, memory, noop
// Only 'langfuse' and 'langsmith' deliver to a vendor; 'console'/'memory' record locally.
```

## Multi-Agent Attribution

Track agent\_id, run\_id, and trace\_id across multi-agent workflows:

```typescript theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
import { Agent, Agents, MemoryObservabilityAdapter, setObservabilityAdapter } from 'praisonai';

const obs = new MemoryObservabilityAdapter();
setObservabilityAdapter(obs);

const researcher = new Agent({
  name: 'Researcher',
  instructions: 'Research topics thoroughly.'
});

const writer = new Agent({
  name: 'Writer', 
  instructions: 'Write clear summaries.'
});

const agents = new AgentTeam({
  agents: [researcher, writer],
  tasks: [
    { agent: researcher, description: 'Research: {topic}' },
    { agent: writer, description: 'Summarize the research' }
  ]
});

await agents.start({ topic: 'AI trends 2024' });

// Traces include agent attribution
const traces = obs.getAllTraces();
for (const trace of traces) {
  console.log(`Agent: ${trace.attribution?.agentId}`);
  console.log(`Run: ${trace.attribution?.runId}`);
}
```

## Environment Variables

Langfuse is the only env-configured adapter that delivers. The rest are read by the SDK for future compatibility \u2014 **setting them today does not enable delivery**; traces still stay in memory.

```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
# Langfuse (delivers when the `langfuse` package is installed)
export LANGFUSE_PUBLIC_KEY=pk-...
export LANGFUSE_SECRET_KEY=sk-...
export LANGFUSE_HOST=https://cloud.langfuse.com

# LangSmith (delivers when LANGSMITH_API_KEY is set)
export LANGSMITH_API_KEY=ls-...                      # or legacy LANGCHAIN_API_KEY
export LANGSMITH_PROJECT=my-project                  # optional; or LANGCHAIN_PROJECT
export LANGSMITH_ENDPOINT=https://api.smith.langchain.com  # optional; or LANGCHAIN_ENDPOINT (http(s) only; http allowed only for loopback)

# The variables below are read but do NOT enable delivery (in-memory only):

# LangWatch
export LANGWATCH_API_KEY=...

# Arize AX (Phoenix)
export ARIZE_API_KEY=...

# Axiom
export AXIOM_TOKEN=...
export AXIOM_DATASET=ai-traces

# Braintrust
export BRAINTRUST_API_KEY=...

# Helicone
export HELICONE_API_KEY=...

# Laminar
export LMNR_PROJECT_API_KEY=...

# Maxim
export MAXIM_API_KEY=...

# Patronus
export PATRONUS_API_KEY=...

# Scorecard
export SCORECARD_API_KEY=...

# SigNoz
export SIGNOZ_ACCESS_TOKEN=...
export SIGNOZ_ENDPOINT=...

# Traceloop
export TRACELOOP_API_KEY=...

# Weave (W&B)
export WANDB_API_KEY=...
export WANDB_PROJECT=my-project
```

## Tool Features Matrix

**Recording ≠ delivery.** Every adapter records Traces/Spans/Events/Errors to memory, but only Langfuse can `Export` (deliver). The `Delivers` column is the one that decides whether traces leave the process. Values match `OBSERVABILITY_TOOLS` in `src/observability/types.ts`.

| Tool | Delivers | Traces | Spans | Events | Errors | Metrics | Export |
| - | - | - | - | - | - | - | - |
| Langfuse | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| LangSmith | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ |
| LangWatch | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |
| Arize | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |
| Axiom | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |
| Braintrust | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |
| Helicone | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |
| Laminar | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |
| Maxim | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |
| Patronus | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |
| Scorecard | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |
| SigNoz | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |
| Traceloop | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |
| Weave | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |
| Console | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |
| Memory | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |

> [!NOTE]
>
> Recording is not delivery. A ✅ in Traces/Spans/Events/Errors means the adapter records it to memory. Only a ✅ in the `Delivers` column means traces reach the vendor \u2014 see the [Delivery Matrix](#observability-delivery-matrix).

## Related

> [card] [Observability CLI](https://praison.ai/docs/js/observability-cli.md)
>
> CLI commands

> [card] [Evaluation](https://praison.ai/docs/js/evaluation.md)
>
> Test Agent quality


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.