Evaluation CLI

CLI commands for evaluation in PraisonAI TypeScript

Evaluation CLI Commands

The praisonai-ts CLI provides the eval command for agent evaluation.

Accuracy Evaluation

# Run accuracy evaluation
praisonai-ts eval accuracy --input "2+2" --expected "4"

# With multiple iterations
praisonai-ts eval accuracy --input "What is 2+2?" --expected "4" --iterations 3 --json

Performance Evaluation

# Run performance benchmark
praisonai-ts eval performance --iterations 10

# With warmup runs
praisonai-ts eval performance --iterations 50 --warmup 5 --json

Reliability Evaluation

# Check tool call reliability
praisonai-ts eval reliability --expected-tools "calculator,web_search"

SDK Usage

For programmatic evaluation:

import { AccuracyEval, PerformanceEval, ReliabilityEval } from 'praisonai';

// Accuracy evaluation
const accuracy = new AccuracyEval({
  agent: myAgent,
  input: "What is 2+2?",
  expectedOutput: "4",
  numIterations: 3
});
const result = await accuracy.run();

// Performance evaluation
const perf = new PerformanceEval({
  func: () => agent.chat("Hello"),
  numIterations: 50,
  warmupRuns: 10
});
const perfResult = await perf.run();

For more details, see the Evaluation SDK documentation.