> ## Documentation Index
> Fetch the complete documentation index at: https://praison.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Knowledge Indexing Errors

> See which files failed to embed instead of getting a silent empty result

When you index a folder into a knowledge base, `Knowledge.add()` tells you which files failed so an outage in your embedding backend never looks like a successful import.

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
graph LR
    subgraph "Indexing a folder"
        A[📁 Directory] --> B[🔍 Walk files]
        B --> C[🧠 Embed each]
        C --> D[✅ results]
        C --> E[❌ errors]
    end

    classDef input fill:#8B0000,stroke:#7C90A0,color:#fff
    classDef process fill:#189AB4,stroke:#7C90A0,color:#fff
    classDef success fill:#10B981,stroke:#7C90A0,color:#fff
    classDef warning fill:#F59E0B,stroke:#7C90A0,color:#fff

    class A input
    class B,C process
    class D success
    class E warning
```

## Quick Start

**1. Simple usage**

Point an agent at a folder and ask a question — indexing happens for you.

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents import Agent

agent = Agent(
    name="Docs Agent",
    instructions="Answer from the docs folder.",
    knowledge=["./docs"],
)

agent.start("What does the knowledge base cover?")
```

**2. Check indexing succeeded**

Call `add()` directly and inspect `result["errors"]` to see which files failed.

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents.knowledge import Knowledge

k = Knowledge()
result = k.add("./docs")

if result["errors"]:
    for e in result["errors"]:
        print(f"failed: {e['file']} — {e['error']}")
else:
    print(f"indexed {len(result['results'])} chunk(s)")
```

***

## How It Works

`Knowledge.add()` walks the directory, embeds each file, and returns the successes and failures together.

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
sequenceDiagram
    participant User
    participant Knowledge
    participant Store

    User->>Knowledge: add("./docs")
    loop Each file
        Knowledge->>Store: store(chunk)
        Store-->>Knowledge: stored id or failure
    end
    Knowledge-->>User: {results, relations, errors}
```

A failed file is recorded instead of dropped, so a directory whose every file fails to embed no longer returns an empty-but-successful result.

***

## Return shape

`add()` on a directory or list always returns these keys — `errors` is present even on success (as an empty list).

| Key | Type | Description |
| - | - | - |
| `results` | `list` | Successfully stored chunk records |
| `relations` | `list` | Graph relations, when a graph store is configured |
| `errors` | `list[dict]` | `{'file': str, 'error': str}` entries, one per file that failed to index |

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
{
    'results': [...],
    'relations': [],
    'errors': [
        {'file': '/abs/path/a.md',
         'error': 'Failed to generate embedding (model=text-embedding-3-small)'},
    ],
}
```

`add([...])` aggregates errors from every input path, so one bad directory never hides errors from another.

> [!NOTE]
>
> `errors` is populated on the **directory** and **list** paths. A single-file `add("./file.md")` where every chunk fails raises `RuntimeError` instead of returning an `errors` list.

***

## Failure modes

> [accordion] All files failed
>
> Every entry lands in `errors` and `results` is empty. The log line names the first cause. Common causes: a wrong embedding model name, a revoked API key, or an exhausted quota.

> [accordion] Partial failure
>
> `results` holds the chunks that succeeded and `errors` names only the ones that failed. The import is not rolled back.

> [accordion] Partial-chunk loss inside one file
>
> When some of a file's chunks embed and others are swallowed by the backend, the file still appears in `results` with the chunks that landed. Only when **every** chunk of a single-file `add()` fails does the call raise (see the next accordion) — a partial loss inside one file does not currently raise or add a per-chunk entry to `errors`.
>
> Guard against silent partial loss by treating a low `results` count for a large file as suspicious, and re-index the file if the embedding backend was degraded during the run.

> [accordion] Single-file add() where every chunk failed
>
> The single-file path raises instead of returning silently:
>
> ```
> RuntimeError: Stored nothing from {path}: all {n} chunk(s) failed (embedding/vector-store backend). See earlier logs.
> ```
>
> Wrap the call in `try` / `except RuntimeError` if the caller wants to keep going.

> [accordion] List input
>
> `add([path1, path2, ...])` aggregates errors from every input path into one `errors` list.

### Durable incremental index

The incremental-index state file (`tracker.json`) is saved atomically as of PR [#5841](https://github.com/MervinPraison/PraisonAI/pull/5841). If a save fails part-way, the previous state file stays intact.

Before, a failed save could wipe the per-file record of stored chunks. The next incremental index would then treat every file as new and re-add it, leaving the old chunks in the vector store, so agents retrieved duplicate or stale content.

As of PR [#6111](https://github.com/MervinPraison/PraisonAI/pull/6111), an unreadable or corrupt state file no longer resets to an empty state silently. `FileTracker.load()` logs a `WARNING` naming the state file and the error, and `FileTracker.save()` logs write failures instead of swallowing them. The empty-state fallback is unchanged — only the diagnostics are new, so a surprise full re-embed is traceable to its cause.

***

## User interaction flow

Bulk-import a docs folder before deployment, then gate the deploy on the result.

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents.knowledge import Knowledge

k = Knowledge()
result = k.add("./docs")

assert not result["errors"], f"{len(result['errors'])} file(s) failed to index"
print(f"indexed {len(result['results'])} chunk(s)")
```

Run this in CI so a broken embedding backend fails the deploy instead of shipping an empty knowledge base.

***

## Using `index()`

`Knowledge.index()` returns an `IndexResult` whose `errors` list carries whole-file failures as `"{filepath}: {message}"` strings.

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents.knowledge import Knowledge

k = Knowledge()
result = k.index("./docs")

if not result.errors:
    print(f"indexed {result.files_indexed} file(s)")
else:
    for err in result.errors:
        print(f"failed: {err}")
```

> [!WARNING]
>
> **`result.success` now reflects `errors`.** `index()` now sets `result.success = not result.errors`. Previously `success` was never assigned, so it kept the dataclass default of `True` — a run where **every** file failed to embed still returned `success=True` with `files_indexed=0` and each failure sitting in `errors`.
>
> If your CI relied on `if result.success` on a pre-PR version, it could report success on a fully failed index. Check `if not result.errors` for behaviour that is correct across versions.

***

## Common Patterns

Fail fast in CI:

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents.knowledge import Knowledge

k = Knowledge()
result = k.add("./docs")
assert not result["errors"]
```

Report and continue — keep what succeeded, log what didn't:

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents.knowledge import Knowledge

k = Knowledge()
result = k.add("./docs")
for e in result["errors"]:
    print(f"skipped: {e['file']} — {e['error']}")
```

Retry only the failed files:

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents.knowledge import Knowledge

k = Knowledge()
result = k.add("./docs")
for e in result["errors"]:
    k.add(e["file"])
```

***

## Best Practices

> [accordion] Always inspect result\['errors'] after a bulk import
>
> A non-empty `errors` list is the only reliable signal that a file was skipped. Check it every time you index a folder.

> [accordion] Treat a non-empty errors list as an outage signal
>
> Failures usually mean the embedding backend is down — a wrong model name, a revoked key, or an exhausted quota. Alert on it rather than logging it as a warning.

> [accordion] Catch RuntimeError for single-file calls
>
> The `errors` list is populated on the directory and list paths. A single-file `add()` where every chunk fails raises `RuntimeError`, so wrap it in `try` / `except` if you need to continue.

> [accordion] Trust errors whether the backend raised or returned empty
>
> `errors` is populated whether the store raised an exception or returned an empty result — embedding backends do both, and both are captured.

***

## Related

> [card] [Knowledge](https://praison.ai/docs/concepts/knowledge.md)
>
> Core knowledge concepts and how agents use a knowledge base.

> [card] [Knowledge Storage](https://praison.ai/docs/features/knowledge-storage.md)
>
> Where knowledge persists and how to change the location.

> [card] [Knowledge Backends](https://praison.ai/docs/features/knowledge-backends.md)
>
> Choose and configure the vector store behind your knowledge base.

> [card] [Incremental Indexing](https://praison.ai/docs/features/incremental-indexing.md)
>
> See also — `index()` returns an `IndexResult` whose `success` and `errors` surface the same per-file failures for `index()` callers.

> [card] [RAG CLI](https://praison.ai/docs/rag/cli.md#index)
>
> The `rag index` wrapper surfaces these errors as a non-zero exit code.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.