> ## Documentation Index
> Fetch the complete documentation index at: https://praison.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Voice Reply (Text-to-Speech)

> Speak the agent's reply back as a native voice note on Telegram, Slack, WhatsApp, and Discord

Speak the agent's plain-text reply back as a native voice note — the outbound counterpart to inbound speech-to-text, off by default and opt-in per channel.

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents import Agent

# 1. Author the agent exactly as before — voice is a transport concern.
agent = Agent(
    name="assistant",
    instructions="Be helpful and concise.",
)

# 2. Enable voice replies on your Telegram channel in gateway.yaml:
#
#    channels:
#      telegram:
#        platform: telegram
#        token: ${TELEGRAM_BOT_TOKEN}
#        stt:   true                    # user speaks in
#        voice: { mode: match_inbound } # bot speaks back when they do
#
# 3. Send a voice note to the bot on Telegram — you get a spoken reply.
#    Send a text message — you get a text reply.
```

One YAML line closes the voice loop — the bot listens **and** speaks.

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
graph LR
    subgraph "Voice Reply Flow"
        User[🎤 User voice note] --> STT[🎙️ STT]
        STT --> Agent[🧠 Agent]
        Agent --> Text[💬 Text reply]
        Text --> TTS[🔊 TTS synth]
        TTS --> Voice[🎧 Voice note reply]
    end

    classDef input fill:#8B0000,stroke:#7C90A0,color:#fff
    classDef process fill:#189AB4,stroke:#7C90A0,color:#fff
    classDef output fill:#10B981,stroke:#7C90A0,color:#fff

    class User input
    class STT,Agent,Text,TTS process
    class Voice output
```

> [!NOTE]
>
> Voice reply mirrors inbound [Voice Notes (Speech-to-Text)](https://praison.ai/docs/features/gateway-stt.md), but it is **off by default**. Set `voice.mode: match_inbound` for the intuitive "speak in / speak out" symmetry.

## Quick Start

**1. Level 1 — Bool shorthand**

`voice: true` turns replies into voice notes for every message (`mode: always`).

```yaml theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
channels:
  telegram:
    platform: telegram
    token: ${TELEGRAM_BOT_TOKEN}
    voice: true    # every reply is also spoken
```

**2. Level 2 — Mode shorthand**

A bare `mode` implies `enabled: true`. Use `match_inbound` to speak back only when the user sent a voice memo.

```yaml theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
channels:
  telegram:
    platform: telegram
    token: ${TELEGRAM_BOT_TOKEN}
    voice:
      mode: match_inbound    # speak back only when spoken to
```

**3. Level 3 — Dict**

Tune the voice, model, speed, format, and length cap.

```yaml theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
channels:
  telegram:
    platform: telegram
    token: ${TELEGRAM_BOT_TOKEN}
    voice:
      enabled: true
      mode: match_inbound
      model: openai/tts-1
      voice: nova
      speed: 1.0
      format: ogg
      max_chars: 4000
```

**4. Level 4 — TtsConfigSchema**

Build the channel config in Python with the validated schema.

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonai_bot.bots._config_schema import (
    ChannelConfigSchema,
    TtsConfigSchema,
)

channel = ChannelConfigSchema(
    platform="telegram",
    token="${TELEGRAM_BOT_TOKEN}",
    voice=TtsConfigSchema(
        enabled=True,
        mode="match_inbound",
        model="openai/tts-1",
        voice="nova",
        speed=1.0,
        format="ogg",
        max_chars=4000,
    ),
)
```

> [!TIP]
>
> The `tts:` block is an accepted alias for `voice:` anywhere. `voice: true` is shorthand for `{enabled: true, mode: always}`.

***

## How It Works

The gateway synthesises the reply after the text is already sent, so voice is always a bonus — never a blocker.

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
sequenceDiagram
    participant User as 🎤 User
    participant Adapter as 📱 Adapter
    participant Agent as 🧠 Agent
    participant TTS as 🔊 TTS

    User->>Adapter: Message (voice or text)
    Adapter->>Agent: chat(text)
    Agent-->>Adapter: 💬 Plain-text reply
    Adapter-->>User: 💬 Text reply (always first)
    alt should_voice_reply == true
        Adapter->>TTS: synthesize_voice_reply(text, cfg)
        alt Synthesis succeeds
            TTS-->>Adapter: 🎧 voice-note file
            Adapter-->>User: 🎧 Voice note (bonus)
        else Skip / fail
            TTS-->>Adapter: None (logged)
        end
    end
```

| Step | What happens |
| - | - |
| **Reply** | The agent returns plain text; the text reply is delivered first, every time. |
| **Gate** | `should_voice_reply` checks `enabled` + `mode` (and, for `match_inbound`, whether the inbound was a voice memo). |
| **Synthesise** | `synthesize_voice_reply` calls `tts_tool` → `AudioAgent.speech` (default `openai/tts-1`). |
| **Deliver** | The adapter sends the file as a native voice note (Telegram `send_voice`). |
| **Degrade** | Any skip or failure logs and is dropped — the text reply already reached the user. |

### Graceful degradation

Voice reply is best-effort — the text reply is always delivered first.

* Empty / whitespace text → skipped silently.
* Reply longer than `max_chars` → skipped, `INFO` log entry.
* TTS tool import fails (missing `openai`/`litellm` deps) → skipped, `WARNING` log entry.
* Synthesis error → skipped, `ERROR` log entry.
* Adapter `send_voice` fails → logged; the text reply already went out.

***

## Configuration Options

Fields from `TtsConfig` / `TtsConfigSchema`.

| Option | Type | Default | Description |
| - | - | - | - |
| `enabled` | `bool` | `False` | Master switch. Off by default (opt-in) — the mirror of STT's on-by-default inbound transcription. |
| `mode` | `str` | `"off"` | Delivery mode: `"off"` \| `"always"` \| `"match_inbound"`. Ignored when `enabled` is `False`. |
| `model` | `Optional[str]` | `None` | Optional TTS model override. `None` uses `openai/tts-1`. |
| `voice` | `Optional[str]` | `None` | Optional voice name (`"alloy"`, `"echo"`, `"fable"`, `"onyx"`, `"nova"`, `"shimmer"`). |
| `speed` | `Optional[float]` | `None` | Speaking-rate multiplier (`0.25`–`4.0`). Provider default when `None`. |
| `format` | `str` | `"ogg"` | Output audio format. `"ogg"` / `"opus"` are the voice-note native formats. |
| `max_chars` | `int` | `4000` | Skip TTS for replies longer than this. `0` disables the cap. |
| `stream` | `bool` | `False` | Opt-in sentence-level streaming. When `True`, the reply is split into sentences and each is synthesised & delivered as it becomes available — time-to-first-audio drops from full-response latency to first-sentence latency. Falls back to a single whole-clip synth when off (default), when the reply is a single sentence, or when the reply is over `max_chars`. |

### Bool shorthand

```yaml theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
channels:
  telegram:
    voice: true    # equivalent to {enabled: true, mode: always}
```

### Resolution order

The effective policy is resolved from, in order:

1. `config.metadata["voice"]` (operator override),
2. `config.metadata["tts"]` (alias),
3. a direct `config.voice` attribute (schema-backed configs),
4. a direct `config.tts` attribute, then
5. the off-by-default `TtsConfig()`.

***

## Choosing a Mode

Pick the mode that matches how chatty your users want the bot to be.

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
graph TB
    Q{Reply in voice?} -->|no| Off["voice: false<br/>(mode: off)"]
    Q -->|yes| Sym{Speak back only<br/>when spoken to?}
    Sym -->|yes| Match["voice:<br/>mode: match_inbound"]
    Sym -->|no| Always["voice:<br/>mode: always"]

    classDef question fill:#F59E0B,stroke:#7C90A0,color:#fff
    classDef opt fill:#6366F1,stroke:#7C90A0,color:#fff

    class Q,Sym question
    class Off,Match,Always opt
```

| Mode | When to use |
| - | - |
| `off` | Default. Never reply in voice. |
| `match_inbound` | **Recommended.** Reply in voice only when the user sent a voice memo — the natural mirror of STT. |
| `always` | Every reply is also a voice note — best for voice-first or accessibility bots. |

> [!TIP]
>
> `mode: match-inbound` (hyphen) normalises to `match_inbound` automatically.

***

## Sentence-Level Streaming

Streaming TTS synthesises and delivers your reply sentence-by-sentence, so the user hears the first clause while the rest is still being spoken.

With `stream: false` (default) the whole reply is synthesised into one clip before anything plays — the user waits for full-response latency. With `stream: true` time-to-first-audio drops to first-sentence latency.

```yaml YAML theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
channels:
  telegram:
    platform: telegram
    token: ${TELEGRAM_BOT_TOKEN}
    voice:
      mode: match_inbound
      stream: true      # NEW: synthesise & deliver sentence-by-sentence
```

```python Python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonai_bot.bots._config_schema import (
    ChannelConfigSchema,
    TtsConfigSchema,
)

channel = ChannelConfigSchema(
    platform="telegram",
    token="${TELEGRAM_BOT_TOKEN}",
    voice=TtsConfigSchema(
        enabled=True,
        mode="match_inbound",
        stream=True,
    ),
)
```

### When streaming kicks in

Streaming is used only when all three preconditions hold — otherwise the whole-file path runs.

* `stream: true` is set.
* The full reply is under `max_chars` (the cap is checked against the whole reply before the per-sentence loop, so long replies can't sneak through as many under-cap clauses).
* The reply splits into more than one sentence. A single-sentence reply falls through to the whole-file path — streaming buys nothing there.

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
sequenceDiagram
    participant User as 🎤 User
    participant Adapter as 📱 Adapter
    participant Agent as 🧠 Agent
    participant TTS as 🔊 TTS

    User->>Adapter: Voice / text message
    Adapter->>Agent: chat(text)
    Agent-->>Adapter: 💬 Plain-text reply
    Adapter-->>User: 💬 Text reply (always first)
    loop for each sentence
        Adapter->>TTS: synthesize(sentence)
        alt success
            TTS-->>Adapter: 🎧 clip
            Adapter-->>User: 🎧 Voice clip
        else fail, nothing played
            TTS-->>Adapter: None → fall back to whole-file
        else fail, audio already playing
            TTS-->>Adapter: None → skip this clause, continue
        end
    end
```

### Failure semantics

Failures are audible-output-aware, so the user never hears a reply restart.

* Nothing played yet + the first clip fails → falls back to whole-file synth on the full text.
* Some audio already played + a clause fails → skip that clause and continue; **never** replays from the beginning.

### When to enable

Flip `stream: true` when replies are usually multi-sentence and users are waiting for audio.

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
graph TB
    Q{Reply is likely long<br/>and voice matters?} -->|no| Off[stream: false<br/>whole-clip synth]
    Q -->|yes| Chatty{Want the user to hear<br/>the first sentence ASAP?}
    Chatty -->|yes| Stream[stream: true]
    Chatty -->|no| Off

    classDef question fill:#F59E0B,stroke:#7C90A0,color:#fff
    classDef opt fill:#6366F1,stroke:#7C90A0,color:#fff

    class Q,Chatty question
    class Off,Stream opt
```

***

## Per-Platform YAML

Telegram is wired today; other adapters share the same `_tts` helper and can adopt it without further core changes.

```yaml Telegram theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
channels:
  telegram:
    platform: telegram
    token: ${TELEGRAM_BOT_TOKEN}
    stt: true                  # user speaks in
    voice:
      mode: match_inbound      # bot speaks back when they do
      voice: nova
      stream: true             # sentence-level delivery (opt-in)
```

```yaml Slack (adapter adoption pending) theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
channels:
  slack:
    platform: slack
    token: ${SLACK_BOT_TOKEN}
    app_token: ${SLACK_APP_TOKEN}
    voice:
      mode: always
```

```yaml WhatsApp (adapter adoption pending) theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
channels:
  whatsapp:
    platform: whatsapp
    token: ${WHATSAPP_TOKEN}
    voice:
      mode: match_inbound
```

```yaml Discord (adapter adoption pending) theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
channels:
  discord:
    platform: discord
    token: ${DISCORD_BOT_TOKEN}
    voice:
      mode: always
```

***

## Platform Support

| Platform | Voice reply status | Notes |
| - | - | - |
| Telegram | ✅ Wired | Reference consumer — delivers via `send_voice`. Optional sentence-level streaming (`stream: true`) delivers a clip per sentence. |
| Slack | 🔜 Adapter can adopt | Same shared `_tts` helper; adapter update needed. |
| WhatsApp | 🔜 Adapter can adopt | Same shared `_tts` helper; adapter update needed. |
| Discord | 🔜 Adapter can adopt | Same shared `_tts` helper; adapter update needed. |

The `[[audio_as_voice]]` manual escape hatch continues to work for pre-rendered audio on Telegram.

***

## User Interaction Flow

A voice-in / voice-out conversation on Telegram with `mode: match_inbound`.

1. The user taps 🎙️ and records a voice note on Telegram.
2. Inbound STT transcribes it → the agent processes the text → produces a plain-text reply.
3. The gateway sees `voice.mode: match_inbound` and that the inbound was a voice memo → synthesises the reply with `openai/tts-1`.
4. The user receives **both**: the text reply and a native voice note they can play right in the chat.
5. If the user next sends a text message, they get only a text reply — no unnecessary audio.
6. When `stream: true` is set, the user hears the first sentence as soon as it is synthesised while the rest of the reply is still being spoken.

***

## Best Practices

> [accordion] Prefer match\_inbound over always
>
> `match_inbound` speaks back only when the user spoke first — the symmetry users expect. Reserve `always` for voice-first or accessibility bots.
>
> ```yaml theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
> voice:
>   mode: match_inbound
> ```

> [accordion] Cap length for chatty agents
>
> Set `max_chars` low so long analytical replies don't become multi-minute audio clips. The text reply still carries the full answer.
>
> ```yaml theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
> voice:
>   mode: match_inbound
>   max_chars: 600
> ```

> [accordion] Ship your TTS provider's API key
>
> `openai/tts-1` is the default, so set `OPENAI_API_KEY`. Any LiteLLM-supported TTS model works via the `model:` field.

> [accordion] Trust the graceful-degradation contract
>
> Voice reply is best-effort — the plain-text reply is always delivered first, so users never lose a message even when synthesis fails.

> [accordion] Stream long replies for lower time-to-first-audio
>
> Set `stream: true` when replies are typically more than one sentence and users are waiting for audio. Time-to-first-audio drops from full-response latency to first-sentence latency, and the whole-file path is still used automatically for single-sentence replies or replies over `max_chars`.
>
> ```yaml theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
> voice:
>   mode: match_inbound
>   stream: true
> ```

> [accordion] Keep the manual escape hatch for pre-rendered audio
>
> The `[[audio_as_voice]]` marker still works on Telegram for audio you've already produced — e.g. a recording pulled from a knowledge base.

***

## Related

> [card] [Voice Notes (Speech-to-Text)](https://praison.ai/docs/features/gateway-stt.md)
>
> The inbound counterpart — transcribe voice notes into text for the agent.

> [card] [Audio Tools](https://praison.ai/docs/features/audio-tools.md)
>
> `tts_tool`, `stt_tool`, and `AudioAgent.speech`.

> [card] [Voice Notes](https://praison.ai/docs/features/gateway-voice-notes.md)
>
> The inbound voice-notes doc for gateway bots.

> [card] [Streaming Replies](https://praison.ai/docs/features/bot-streaming-replies.md)
>
> The text-streaming counterpart — stream the reply text into chat as it is generated.

> [card] [Gateway](https://praison.ai/docs/gateway.md)
>
> The overall gateway pattern — voice reply is one of several channel features.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.