Streaming Tracker • AI Agents Framework

StreamingTracker: Track streaming operations (LLM responses).

StreamingTracker

Defined in the profiler module.

AI Agents Framework

Track streaming operations (LLM responses).

Constructor

name · str · required

No description available.

Methods

start()

Start tracking.

first_token()

Mark time to first token.

chunk()

Record a chunk received.

end()

End tracking and record to Profiler.

ttft_ms()

Get time to first token in ms.

elapsed_ms()

Get elapsed time in ms.

Usage

tracker = StreamingTracker("chat")
    tracker.start()
    tracker.first_token()  # Mark TTFT
    for chunk in stream:
        tracker.chunk()
    tracker.end(total_tokens=100)

Source

View on GitHub

praisonai/profiler.py at line 142