Skip to main content

​
Observability CLI Commands

The praisonai-ts CLI provides comprehensive commands for managing 14+ observability integrations.

​
Command Overview

CommandDescription
observability listList all 14+ available tools
observability doctorCheck environment variables for all tools
observability doctor <tool>Check specific tool setup
observability test <tool>Test tool with a trace
observability infoShow observability feature information

​
List Observability Tools

# List all available tools
praisonai-ts observability list

# List with verbose output (shows env vars)
praisonai-ts observability list --verbose

# Get JSON output for CI/CD
praisonai-ts observability list --json
Example Output: 
Observability Tools

  Total: 17 tools (3 ready)

  Built-in (no setup required):
    ✅ console      Console logging adapter
    ✅ memory       In-memory storage adapter
    ✅ noop         No-op adapter (zero overhead)

  External Integrations:
    ⚠️ langfuse     Langfuse LLM observability
    ⚠️ langsmith    LangChain observability platform
    ⚠️ langwatch    LangWatch monitoring
    ⚠️ arize        Arize AX Phoenix observability
    ⚠️ axiom        Axiom log analytics
    ⚠️ braintrust   Braintrust AI evaluation
    ⚠️ helicone     Helicone LLM proxy
    ⚠️ laminar      Laminar AI observability
    ⚠️ maxim        Maxim AI testing
    ⚠️ patronus     Patronus AI evaluation
    ⚠️ scorecard    Scorecard AI testing
    ⚠️ signoz       SigNoz OpenTelemetry
    ⚠️ traceloop    Traceloop OpenLLMetry
    ⚠️ weave        Weights & Biases Weave
JSON Output: 
{
  "success": true,
  "data": {
    "providers": [
      { "name": "langfuse", "description": "Langfuse LLM observability", "hasEnvKey": false },
      { "name": "langsmith", "description": "LangChain observability platform", "hasEnvKey": false },
      { "name": "console", "description": "Console logging adapter", "hasEnvKey": true }
    ],
    "total": 17,
    "ready": 3,
    "builtin": 3,
    "external": 14
  }
}

​
Observability Doctor

Check environment setup for tools: 
# Check all tools
praisonai-ts observability doctor

# Check specific tool
praisonai-ts observability doctor langfuse
praisonai-ts observability doctor langsmith
praisonai-ts observability doctor arize

# JSON output
praisonai-ts observability doctor langfuse --json
Example Output (specific tool): 
Observability Doctor: langfuse
  Description: Langfuse LLM observability
  Package: langfuse
  Environment Variable: LANGFUSE_SECRET_KEY
  Status: ❌ Missing API Key

  Set the API key with:
    export LANGFUSE_SECRET_KEY=your-api-key

  Features:
    Traces: ✅  Spans: ✅  Events: ✅
    Errors: ✅  Metrics: ✅  Export: ✅
Example Output (all tools): 
Observability Environment Check

  Total Tools: 17
  Ready: 3 ✅

  Ready Tools:
    ✅ console
    ✅ memory
    ✅ noop

​
Observability Test

Test a tool with a real trace: 
# Test built-in tools (no API key needed)
praisonai-ts observability test memory
praisonai-ts observability test console

# Test external tools (requires API key)
praisonai-ts observability test langfuse
praisonai-ts observability test langsmith

# JSON output
praisonai-ts observability test memory --json
Example Output: 
Testing memory observability...

  ✅ Test Passed
  Tool: memory
  Latency: 2ms
  Trace ID: abc123xyz
JSON Output: 
{
  "success": true,
  "data": {
    "tool": "memory",
    "status": "success",
    "latency_ms": 2,
    "trace_id": "abc123xyz"
  }
}

​
Observability Info

Get feature information: 
praisonai-ts observability info
praisonai-ts observability info --json

​
Options Matrix

OptionDescriptionDefault
--jsonOutput in JSON formatfalse
--verboseShow detailed outputfalse
--output <format>Output format (json/text/pretty)pretty
--tool <name>Specify tool-

​
Available Tools

ToolDescriptionEnv Variable
langfuseLangfuse LLM observabilityLANGFUSE_SECRET_KEY
langsmithLangChain observabilityLANGCHAIN_API_KEY
langwatchLangWatch monitoringLANGWATCH_API_KEY
arizeArize AX PhoenixARIZE_API_KEY
axiomAxiom log analyticsAXIOM_TOKEN
braintrustBraintrust AI evaluationBRAINTRUST_API_KEY
heliconeHelicone LLM proxyHELICONE_API_KEY
laminarLaminar AI observabilityLMNR_PROJECT_API_KEY
maximMaxim AI testingMAXIM_API_KEY
patronusPatronus AI evaluationPATRONUS_API_KEY
scorecardScorecard AI testingSCORECARD_API_KEY
signozSigNoz OpenTelemetrySIGNOZ_ACCESS_TOKEN
traceloopTraceloop OpenLLMetryTRACELOOP_API_KEY
weaveWeights & Biases WeaveWANDB_API_KEY
consoleConsole logging(none)
memoryIn-memory storage(none)
noopNo-op (zero overhead)(none)

​
Environment Variables

# Langfuse
export LANGFUSE_PUBLIC_KEY=pk-...
export LANGFUSE_SECRET_KEY=sk-...
export LANGFUSE_HOST=https://cloud.langfuse.com

# LangSmith
export LANGCHAIN_API_KEY=ls-...
export LANGCHAIN_PROJECT=my-project

# LangWatch
export LANGWATCH_API_KEY=...

# Arize AX (Phoenix)
export ARIZE_API_KEY=...

# Axiom
export AXIOM_TOKEN=...
export AXIOM_DATASET=ai-traces

# Braintrust
export BRAINTRUST_API_KEY=...

# Helicone
export HELICONE_API_KEY=...

# Laminar
export LMNR_PROJECT_API_KEY=...

# Maxim
export MAXIM_API_KEY=...

# Patronus
export PATRONUS_API_KEY=...

# Scorecard
export SCORECARD_API_KEY=...

# SigNoz
export SIGNOZ_ACCESS_TOKEN=...
export SIGNOZ_ENDPOINT=...

# Traceloop
export TRACELOOP_API_KEY=...

# Weave (W&B)
export WANDB_API_KEY=...
export WANDB_PROJECT=my-project

​
Troubleshooting

​
Missing API Key

# Check which keys are missing
praisonai-ts observability doctor

# Check specific tool
praisonai-ts observability doctor langfuse
# Output: ❌ Missing API Key
# Set it with: export LANGFUSE_SECRET_KEY=your-api-key

​
Tool Not Available

# Check if tool package is installed
praisonai-ts observability doctor langfuse
# If package missing, install with:
# npm install langfuse

​
Test Failures

# Run test with verbose output
praisonai-ts observability test langfuse --verbose

# Check JSON error details
praisonai-ts observability test langfuse --json

​
SDK Usage

For programmatic observability: 
import { 
  createObservabilityAdapter, 
  setObservabilityAdapter,
  OBSERVABILITY_TOOLS,
  listObservabilityTools 
} from 'praisonai';

// List all tools
const tools = listObservabilityTools();
console.log(tools.map(t => t.name)); // ['langfuse', 'langsmith', ...]

// Create adapter
const obs = await createObservabilityAdapter('langfuse');
setObservabilityAdapter(obs);

// Use with agents
const agent = new Agent({
  instructions: 'You are helpful.'
});
await agent.chat('Hello');

// Flush traces
await obs.flush();