LLM Module

LLM client and model utilities for agent interactions

LLM Module

The LLM module provides the core language model client and utilities for agent interactions, supporting multiple providers through LiteLLM.

Param Cluster Map

Cluster	Legacy Params	Consolidated To	Status
LLM	llm, llm_config, function_calling_llm	llm=	✅ Done

Note: base_url and api_key remain separate parameters (connection/auth constraint).

Precedence Ladder

Instance > Config > Array > Dict > String > Bool > Default

Ways to Use llm=

Form	Example	Description
String	llm="gpt-4o"	Model name
Alias	model="gpt-4o"	Same as llm=
Dict	llm={"model": "gpt-4o", "temperature": 0.7}	Config dict
Instance	llm=LLM(model="gpt-4o")	LLM instance

Legacy (Deprecated)

# ⚠️ DEPRECATED - still works but emits warning
Agent(llm={"temperature": 0.7})
Agent(function_calling_llm="gpt-4o-mini")

# ✅ NEW - use llm= instead
Agent(llm="gpt-4o")
Agent(model="gpt-4o")  # alias

Installation

pip install praisonaiagents

Quick Start

from praisonaiagents.llm import LLM

llm = LLM(model="gpt-4o-mini")
response = llm.chat("What is machine learning?")
print(response)

Classes

LLM

Main LLM client class supporting multiple providers.

from praisonaiagents.llm import LLM

llm = LLM(
    model="gpt-4o-mini",
    temperature=0.7,
    max_tokens=1000
)

Constructor

Parameter	Type	Default	Description
model	str	"gpt-4o-mini"	Model identifier
temperature	float	0.7	Sampling temperature
max_tokens	int	None	Maximum tokens in response
api_key	str	None	API key (uses env var if not set)
base_url	str	None	Custom API base URL

Methods

Method	Description
chat(prompt, **kwargs)	Send a chat message
chat_async(prompt, **kwargs)	Async chat message
stream(prompt, **kwargs)	Stream response
get_response(messages, **kwargs)	Get response from messages

LLMContextLengthExceededException

Exception raised when context length is exceeded.

from praisonaiagents.llm import LLMContextLengthExceededException

try:
    response = llm.chat(very_long_prompt)
except LLMContextLengthExceededException as e:
    print(f"Context too long: {e}")

OpenAIClient

OpenAI-compatible client for direct API access.

from praisonaiagents.llm import OpenAIClient, get_openai_client

client = get_openai_client(model="gpt-4o-mini")
response = client.chat.completions.create(
    messages=[{"role": "user", "content": "Hello"}]
)

ModelRouter

Intelligent model routing based on task complexity.

from praisonaiagents.llm import ModelRouter, TaskComplexity

router = ModelRouter(
    models={
        TaskComplexity.LOW: "gpt-4o-mini",
        TaskComplexity.MEDIUM: "gpt-4o",
        TaskComplexity.HIGH: "gpt-4-turbo"
    }
)

model = router.select(task="Simple greeting")

TaskComplexity

Level	Description
LOW	Simple tasks, quick responses
MEDIUM	Moderate complexity
HIGH	Complex reasoning, long context

ModelProfile

Profile for a model’s capabilities.

from praisonaiagents.llm import ModelProfile

profile = ModelProfile(
    name="gpt-4o-mini",
    context_length=128000,
    supports_tools=True,
    supports_vision=True,
    cost_per_1k_input=0.00015,
    cost_per_1k_output=0.0006
)

Response Types

ChatCompletion

from praisonaiagents.llm import ChatCompletion

# Response structure
completion.id           # Completion ID
completion.choices      # List of choices
completion.usage        # Token usage
completion.model        # Model used

ChatCompletionMessage

from praisonaiagents.llm import ChatCompletionMessage

message.role      # "assistant", "user", etc.
message.content   # Message content
message.tool_calls  # Tool calls if any

ToolCall

from praisonaiagents.llm import ToolCall

tool_call.id        # Tool call ID
tool_call.type      # "function"
tool_call.function  # Function details

CompletionUsage

from praisonaiagents.llm import CompletionUsage

usage.prompt_tokens      # Input tokens
usage.completion_tokens  # Output tokens
usage.total_tokens       # Total tokens

Utility Functions

supports_structured_outputs

Check if a model supports structured outputs.

from praisonaiagents.llm import supports_structured_outputs

if supports_structured_outputs("gpt-4o-mini"):
    # Use structured output mode
    pass

supports_streaming_with_tools

Check if a model supports streaming with tool calls.

from praisonaiagents.llm import supports_streaming_with_tools

if supports_streaming_with_tools("gpt-4o"):
    # Enable streaming with tools
    pass

process_stream_chunks

Process streaming response chunks.

from praisonaiagents.llm import process_stream_chunks

for chunk in process_stream_chunks(stream_response):
    print(chunk.content, end="")

create_routing_agent

Create an agent with model routing.

from praisonaiagents.llm import create_routing_agent

agent = create_routing_agent(
    name="Smart Agent",
    router=model_router
)

Usage Examples

Basic Chat

from praisonaiagents.llm import LLM

llm = LLM(model="gpt-4o-mini")
response = llm.chat("Explain quantum computing in simple terms")
print(response)

Streaming Response

from praisonaiagents.llm import LLM

llm = LLM(model="gpt-4o-mini")
for chunk in llm.stream("Write a short story"):
    print(chunk, end="", flush=True)

With Tools

from praisonaiagents.llm import LLM

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string"}
                },
                "required": ["location"]
            }
        }
    }
]

llm = LLM(model="gpt-4o-mini")
response = llm.chat(
    "What's the weather in Tokyo?",
    tools=tools
)

Model Routing

from praisonaiagents.llm import ModelRouter, TaskComplexity

router = ModelRouter(
    models={
        TaskComplexity.LOW: "gpt-4o-mini",
        TaskComplexity.MEDIUM: "gpt-4o",
        TaskComplexity.HIGH: "claude-3-opus"
    }
)

# Router automatically selects appropriate model
model = router.select(task="Complex mathematical proof")
llm = LLM(model=model)

Different Providers

from praisonaiagents.llm import LLM

# OpenAI
openai_llm = LLM(model="gpt-4o-mini")

# Anthropic
claude_llm = LLM(model="claude-3-sonnet-20240229")

# Ollama (local)
ollama_llm = LLM(model="ollama/llama3")

# Google
gemini_llm = LLM(model="gemini/gemini-pro")

# Groq
groq_llm = LLM(model="groq/llama3-70b-8192")

Environment Variables

Variable	Description
OPENAI_API_KEY	OpenAI API key
ANTHROPIC_API_KEY	Anthropic API key
GOOGLE_API_KEY	Google AI API key
GROQ_API_KEY	Groq API key
OPENROUTER_API_KEY	OpenRouter API key

Related

• Agent - Using LLM with agents
• Telemetry - Token tracking