eval • AI Agent SDK

PraisonAI Agents Evaluation Framework.

eval

AI Agent

PraisonAI Agents Evaluation Framework.

Provides comprehensive evaluation capabilities for AI agents with zero performance impact when not in use through lazy loading.

Evaluator Types:

• AccuracyEvaluator: Compare output against expected output using LLM-as-judge
• PerformanceEvaluator: Measure runtime and memory usage
• ReliabilityEvaluator: Verify expected tool calls are made
• CriteriaEvaluator: Evaluate against custom criteria

Example: >>> from praisonaiagents.eval import AccuracyEvaluator >>> evaluator = AccuracyEvaluator( … agent=my_agent, … input_text=“What is 2+2?”, … expected_output=“4” … ) >>> result = evaluator.run(print_summary=True)

Import

from praisonaiagents import eval

Constants

Name	Value
_LAZY_IMPORTS	{'BaseEvaluator': ('base', 'BaseEvaluator'), 'AccuracyEvaluator': ('accuracy', 'AccuracyEvaluator'), 'PerformanceEvaluator': ('performance', 'PerformanceEvaluator'), 'ReliabilityEvaluator': ('reliabil...

---

Related Documentation

Evaluation Concept

LLM as Judge

Evaluation Loop