Degraded Channel Isolation

One channel’s missing credential no longer takes the whole gateway down — that channel is isolated as degraded, the others keep serving

One channel’s unavailable credential isolates just that channel as degraded — every healthy channel keeps serving. The same surface now covers two cases: a token that was unavailable at boot (this page), and a token the platform rejected at runtime (see Runtime credential rejection).

from praisonaiagents import Agent

agent = Agent(name="assistant", instructions="Be helpful.")
# Rotate Slack's token overnight — Telegram + Discord keep serving; Slack is
# marked degraded until the new token is available.
agent.start("Anyone there?")

Before this change one channel’s ValueError aborted the whole GatewayConfigSchema, taking every channel offline. Now the schema returns an empty token for the unavailable credential, the runtime’s pre-existing “no token → skip” path isolates that one channel, and health() reports it as degraded.

This is a resilience contract, not a new API. Nothing changes in how you write gateway.yaml — the same Secret References you already use now fail open per-channel instead of failing closed for the whole gateway.

---

What Changed

A single channel’s unavailable credential is isolated; structural errors and the gateway’s own ingress secret still fail closed.

Scenario	Before	After
Single channel token env var unset at boot	GatewayConfigSchema raised; all channels dead	Only that channel is skipped and marked degraded; siblings keep serving
{source: file, id: /run/secrets/x} file empty at boot	Same — whole gateway down	Same isolation as above
Structurally invalid config (missing platform:)	Fails closed	Still fails closed (unchanged)
Gateway’s own ingress / auth secret unavailable	Fails closed	Still fails closed (unchanged)
Preflight probe running (--preflight, default)	Hard-abort on failure	Unchanged — preflight still hard-aborts. Isolation only kicks in when preflight is skipped or a credential drops between preflight and full config load
Token accepted at boot, rejected mid-flight (revoked/rotated/expired → 401/403)	Whole channel went to FAILED; required a process restart to try a new token	Isolated as CREDENTIAL_UNAVAILABLE (degraded); reconnect() / hot-reload picks up the fresh token without a restart. See Channel Supervision.

The single most important distinction: a channel credential fails open (isolated, degraded); the gateway’s own config and ingress secret fail closed (abort).

---

How to Observe the Degraded State

The degraded channel is visible in the startup log, gateway doctor, channels --probe, and GET /health — no value is ever printed.

Startup log

One redacted WARNING is logged per degraded channel. An unset ${ENV} token logs:

Channel token env var 'SLACK_BOT_TOKEN' not set — channel will be skipped (degraded). Set it with: export SLACK_BOT_TOKEN=your_token

A {source, id} reference that resolves to configured-but-unavailable / missing logs:

Channel secret configured-unavailable — channel will be skipped (degraded): env var 'SLACK_BOT_TOKEN' empty

The token value is never printed — resolved secrets are registered for redaction.

praisonai gateway doctor

The secrets block reports configured-but-unavailable or missing for the affected field.

$ praisonai gateway doctor
Credential availability (values never shown):
telegram     token         ✓  available
slack        token         ✓  available
slack        app_token     ✗  configured-but-unavailable

See Credential availability (without revealing values) for the full table.

praisonai gateway channels --probe

The channel listing surfaces the same per-channel verdict — a one-line probe you can run against a live gateway.

GET /health on the running gateway

The degraded channel appears under channels, distinguished from a running channel by status: "degraded" and a reason.

{
  "status": "healthy",
  "channels": {
    "telegram": { "platform": "telegram", "running": true },
    "discord":  { "platform": "discord",  "running": true },
    "slack":    {
      "platform": "slack",
      "running": false,
      "status": "degraded",
      "reason": "credential unavailable"
    }
  }
}

A degraded channel stays queryable in health() — it does not silently vanish, so a monitor can tell “configured-but-unavailable” apart from “never configured”.

The same shape is used whether the credential was unavailable at boot or rejected at runtime — an operator dashboard doesn’t need two code paths. A runtime-rejected channel adds a supervision block with state: "credential-unavailable"; see Runtime credential rejection.

---

How to Recover

1. Fix the credential

   Rotate the env var, refill the mounted secret file, or restore the secret manager so the token resolves to available again.

2. Reload the config

   An empty credential is resolved once at config load, so praisonai gateway reconnect <channel> alone does not re-read it — reconnect drives the channel supervisor, not the secret resolver. Trigger a config reload (or restart) so the schema re-resolves the now-available token. See Channel Supervision for reconnect/pause/resume.

3. Confirm it recovered

   On the next config load, a channel that previously degraded but now has a token is removed from the degraded set automatically. Verify with praisonai gateway doctor or GET /health.

For a token that was rejected at runtime (rather than empty at boot), praisonai gateway reconnect <channel> is enough on env-var-backed tokens — the supervisor wakes the parked channel and Bot.start() re-resolves the env var. A full config hot-reload is required only when the token lives in a {source: file, id: …} reference that was rewritten on disk while the channel was parked.

Gateway Hot Reload picks the new credential up without a full restart — the reload re-resolves every channel’s secret and drops recovered channels out of the degraded set.

---

When You Do NOT Want This Behaviour

Some operators prefer strict fail-closed — enforce it at the operator layer, not the schema.

Preflight (--preflight, the default) still hard-aborts on a credential failure at start. The isolation contract only matters after the gateway is already running, or when preflight was explicitly skipped with --no-preflight.

There is intentionally no knob to make the schema raise again — that behaviour was removed precisely because it took healthy channels down. To gate a deploy on strict availability, fail the release if praisonai gateway doctor --json reports any configured-but-unavailable or missing:

praisonai gateway doctor --config gateway.yaml --json > doctor.json
python -c "import json,sys; d=json.load(open('doctor.json')); \
  bad=[(c,f,v) for c,fs in d.get('secrets',{}).items() for f,v in fs.items() if v not in ('available','configured')]; \
  sys.exit(1) if bad else None"

---

Best Practices

Gate deploys on praisonai gateway doctor --json

Read the secrets block and fail the deploy if any field is not available (or configured for exec sources). This gives you strict fail-closed at release time while keeping the resilient fail-open behaviour at runtime.

Use {source: file} for K8s / Docker secrets

A mounted secret file survives a container restart; a process env var may not. Cross-link: Secret References. File-sourced credentials keep resolving after a restart, so a channel recovers on the next config load without manual intervention.

Alert on the degraded WARNING log line

The channel will be skipped (degraded) WARNING is your operator canary. Wire it into your alerting so a rotated-but-not-yet-refilled credential pages you before users notice the channel is quiet.

Never rely on preflight alone for continuous safety

Preflight only checks credentials at start. A token can go stale after preflight passes — expiry, rotation, a secret-store blip. Degraded-channel isolation is what keeps the other channels serving when that happens.

---

Related

Gateway Secret References

Load credentials from files, env vars, or secret managers.

Gateway CLI

gateway doctor and channels --probe credential availability.

Gateway Channel Supervision

CREDENTIAL_UNAVAILABLE state and the refresh_credentials() hook.

Gateway Reliability Presets

Drain, admission, and fail-closed opt-outs.