Interactive explainer for agent builders
Why Traditional RAG Breaks at Scale — and How Agentic Document Search Differs
How does an agent actually search over a large, distributed set of documents? Classic RAG chunks documents, embeds them, and grabs the top-k nearest vectors. That works until facts split across chunks, answers need multiple hops, or knowledge lives in separate indexes. Explore each failure mode below.
The RAG Pipeline, Animated
A document is split into fixed-size chunks, each embedded into vector space. Drag the sliders: at small chunk sizes the key fact splits across two chunks.
Embedding Space Simulator
200 chunks from three source documents, projected to 2D. Pick a query: the query point animlands, top-k neighbors light cyan, and relevant-but-distant chunks pulse coral — semantic near-misses. Toggle distributed corpora to see cross-index blind spots.
Worked Example: One Question, Two Strategies
"What changed in the Q3 refund policy and who approved it?"
Naive RAG
- Embed the whole question as one vector
- Top-k hits: refund policy chunks only
- Approval email lives in a second store — never queried
- Answer: policy change described, approver unknown
Agentic Search
- Plan: two sub-questions detected
- Search policy corpus for Q3 changes
- Follow reference to approval thread
- Query the email corpus directly
- Synthesize: change + approver, with citations
Known Limits of Traditional RAG
Chunk boundary loss
Facts spanning a split are never retrieved intact; overlap only patches small spans.
Top-k ceiling
If the answer needs the 26th-nearest chunk, k=25 silently misses it.
Stale indexes
Embeddings freeze at index time; updated documents drift from their vectors.
Multi-hop questions
One query vector cannot express "find X, then follow its reference to Y".
Distributed silos
Separate indexes per team or system mean single-index retrieval is blind by design.