Deduplicate Chunks • Rust AI Agent SDK

deduplicate_chunks: Deduplicate chunks by content similarity.

deduplicate_chunks

Function

This function is defined in the rag module.

Deduplicate chunks by content similarity.

Signature

def deduplicate_chunks(chunks: Vec<ContextChunk>, _threshold: f32) -> Vec<ContextChunk>

Parameters

chunks · Vec<ContextChunk> · required

No description available.

_threshold · f32 · required

No description available.

Returns

Returns · Vec<ContextChunk>

The result of the operation.

Uses

• any
• comparison

Source

View on GitHub

praisonai/src/rag/mod.rs at line 623

---

Related Documentation

Rust Chunking

Rust RAG