Gateway API Endpoints

Serve OpenAI-compatible and MCP HTTP endpoints from the running gateway

Turn the running gateway into an OpenAI-compatible and MCP endpoint so SDK clients and MCP tools reach the same live agents as chat users.

One process, one agent state, three protocols. This is different from praisonai serve openai, which runs a separate standalone OpenAI-only process.

Quick Start

1. Enable in gateway.yaml

   Add the api block to your gateway config:

   gateway:
     api:
       openai: true
       mcp: true
   
2. Start with CLI flags

   Enable the same surfaces from the command line:

   praisonai gateway start --config gateway.yaml --openai-api --mcp
   
3. Enable in Python

   Pass constructor flags to the gateway:

   from praisonai.gateway import WebSocketGateway
   
   WebSocketGateway(config=cfg, openai_api=True, mcp=True).start()
   

---

How It Works

Each API request dispatches into the gateway’s own registered agents, sharing the same session store and admission gate as chat users.

Step	What happens
Resolve agent	The model field selects a registered agent; falls back to the first agent
Reuse session	Each caller gets a stable session keyed by OpenAI-Session / X-Session-Id header or bearer token
Admit turn	The turn passes through the same admission gate as chat users
Respond	The reply is returned in OpenAI or MCP shape

---

Endpoints Exposed

Surface	Method + Path	Notes
OpenAI chat	POST /v1/chat/completions	SSE streaming supported (stream: true); real usage returned; opt-in streamed usage via stream_options.include_usage
OpenAI responses	POST /v1/responses	Accepts string or messages-style input; returns usage in Responses-native shape
OpenAI models	GET /v1/models	Lists gateway-registered agent IDs
MCP JSON-RPC	POST /mcp	Methods: initialize, tools/list, tools/call

Call the OpenAI surface with any standard client:

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8765/v1", api_key="gateway-token")
reply = client.chat.completions.create(
    model="assistant",
    messages=[{"role": "user", "content": "Hello"}],
)
print(reply.choices[0].message.content)

---

Token Usage

Every response reports real per-turn token counts from the agent’s LLM instance.

Each surface reports usage in its own OpenAI-native shape:

Surface	usage shape
/v1/chat/completions	{prompt_tokens, completion_tokens, total_tokens} (OpenAI Chat Completions)
/v1/responses	{input_tokens, output_tokens, total_tokens} (OpenAI Responses-native)

When an agent exposes no LLM metrics, usage falls back to zeros in the correct spec shape, so clients never need to special-case a missing block.

Streaming Usage

Opt in with stream_options.include_usage: true to receive a streamed usage chunk — the same contract OpenAI’s own API uses.

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8765/v1", api_key="gateway-token")
stream = client.chat.completions.create(
    model="assistant",
    messages=[{"role": "user", "content": "Hello"}],
    stream=True,
    stream_options={"include_usage": True},
)
for chunk in stream:
    if chunk.usage:
        print(chunk.usage)  # final chunk: real prompt/completion/total tokens

The usage-only chunk is emitted right before data: [DONE]. Without include_usage, the stream frames are unchanged and no extra chunk is sent.

Streaming is still buffered: content is delivered once the full turn completes, so time-to-first-content matches non-streaming. Only the trailing usage chunk is new — this is not incremental token streaming.

---

Configuration Options

The gateway.api block maps to the ApiConfig dataclass. Both surfaces are opt-in; when both are False (default), no extra routes are mounted.

Option	Type	Default	Description
openai	bool	False	Serve /v1/chat/completions, /v1/responses, /v1/models backed by the gateway’s live agents and sessions.
mcp	bool	False	Serve an MCP JSON-RPC endpoint at /mcp exposing registered agents as callable tools.

ApiConfig also exposes an enabled property (true if either surface is on), plus to_dict() and from_dict().

Full API surface

---

How Auth Works

Every API route is protected by the same gateway.auth_token as /info and /metrics.

The /info endpoint advertises which surfaces are enabled in its api field, so clients can introspect a running gateway before connecting.

---

When to Use vs praisonai serve openai

Use the gateway api: block when you want SDK clients and MCP tools to share live agents and sessions with chat users. Use praisonai serve openai when you want a standalone, lightweight OpenAI-only process with no gateway state. See OpenAI-Compatible Server.

Need	Choose
SDK clients share chat users’ live sessions	Gateway api: block
Standalone OpenAI-only endpoint	praisonai serve openai
Expose gateway agents as MCP tools	Gateway api.mcp

---

Best Practices

Keep surfaces off unless needed

Leave openai: false and mcp: false (the defaults) unless a client needs them. Disabled surfaces mount no routes and leave the gateway unchanged.

Protect with auth_token in production

Every /v1/* and /mcp route uses the same gateway.auth_token. Set a strong token when binding to any non-loopback interface.

Pin conversations with a session header

Pass an OpenAI-Session or X-Session-Id header to reuse one agent session across calls. Without it, stateless callers get a fresh session per request.

Introspect surfaces with /info

GET /info returns an api field listing enabled surfaces, so tooling can confirm what a gateway exposes before dispatching.

Enable streamed usage for cost accounting

Pass stream_options={"include_usage": True} when using stream=True if your client tracks cost per response. The extra chunk arrives right before [DONE] and carries real per-turn totals.

Usage is per-turn, even under concurrent sessions

The gateway snapshots the agent’s per-turn token metrics in the same execution context that produced the reply, so a usage field returned to one caller cannot be corrupted by a concurrent turn on the same shared agent. No configuration required.

---

Related

Gateway

Gateway architecture and YAML configuration

OpenAI-Compatible Server

Standalone OpenAI-only server process

MCP Integration

Model Context Protocol servers and clients

Gateway CLI

CLI commands for managing the gateway