Agentic Evaluator Optimizer

Generate and refine agent outputs through iterative evaluator feedback

A generator agent produces drafts; an evaluator agent scores them and loops until approved — wired with AgentFlow and repeat.

from praisonaiagents import Agent, AgentFlow, repeat

generator = Agent(
    name="Generator",
    instructions="Generate content. Improve based on feedback if provided.",
)
evaluator = Agent(
    name="Evaluator",
    instructions="If good, respond with APPROVED. Otherwise give specific feedback.",
)

def is_approved(ctx) -> bool:
    return "approved" in ctx.previous_result.lower()

workflow = AgentFlow(steps=[generator, repeat(evaluator, until=is_approved, max_iterations=3)])
result = workflow.start("Write a compelling product description for an AI assistant")
print(result["output"][:500])

The user sets a creative task; a generator–evaluator loop repeats until the evaluator approves the output.

Quick Start

1. Generator + evaluator loop

   from praisonaiagents import Agent, AgentFlow, repeat
   
   generator = Agent(
       name="Generator",
       instructions="Generate high-quality content. Improve based on feedback if provided.",
   )
   evaluator = Agent(
       name="Evaluator",
       instructions="Evaluate quality. Respond APPROVED if good; otherwise give specific feedback.",
   )
   
   def is_approved(ctx) -> bool:
       return "approved" in ctx.previous_result.lower()
   
   workflow = AgentFlow(
       steps=[
           generator,
           repeat(evaluator, until=is_approved, max_iterations=3),
       ]
   )
   
   result = workflow.start("Write a compelling product description for an AI assistant")
   print(result["output"][:500])
   
2. Tune the feedback loop

   from praisonaiagents import Agent, AgentFlow, repeat
   
   generator = Agent(name="Generator", instructions="Generate and refine content.")
   evaluator = Agent(
       name="Evaluator",
       instructions="Respond APPROVED when quality meets the bar; otherwise give feedback.",
   )
   
   def is_approved(ctx) -> bool:
       return "approved" in ctx.previous_result.lower()
   
   workflow = AgentFlow(
       steps=[
           generator,
           repeat(evaluator, until=is_approved, max_iterations=5),
       ],
       output="verbose",
   )
   
   result = workflow.start("Draft a one-paragraph privacy policy summary")
   print(result["output"])
   

---

How It Works

The evaluator-optimiser pattern suits content generation, code refinement, and any task where quality improves with structured feedback.

---

Best Practices

Write clear evaluation criteria

Tell the evaluator exactly what APPROVED means — tone, length, format, or test coverage — so the loop converges quickly.

Cap iterations with max_iterations

Set max_iterations on repeat() to avoid runaway loops when the evaluator never approves.

Keep generator instructions feedback-aware

Instruct the generator to incorporate evaluator feedback explicitly on each pass.

Use verbose mode while tuning

Enable verbose output on agents while calibrating criteria, then disable for production runs.

---

Related

Dynamic Judge

Score outputs against custom rubrics

Workflows

Compose multi-step agent workflows with AgentFlow