Interactive explainer for agent builders

Why Traditional RAG Breaks at Scale — and How Agentic Document Search Differs

How does an agent actually search over a large, distributed set of documents? Classic retrieval-augmented generation chunks documents, embeds the chunks, and pulls the top-k nearest vectors. That works on a demo corpus. At scale — many stores, multi-hop questions, stale indexes — it silently misses. Scroll to see exactly where it breaks, then watch an agentic loop fix it.

Step one

The RAG Pipeline, Animated

A document is split into fixed-size chunks, each chunk is embedded, and each embedding lands as a point in vector space. Drag the sliders: at small chunk sizes with low overlap, a key fact gets split across two chunks — and neither chunk alone answers the question.

Step two

Embedding Space Simulator

Two hundred chunks from four documents, projected in 3D (drag to rotate). Pick a query: the query vector flies in, its top-k nearest neighbors light up cyan — and chunks that a human would call relevant, but that sit far away in embedding space, pulse coral. Those are semantic near-misses. Toggle distributed corpora to see how separate indexes create blind spots.

Policy docsEmail archiveWikiTicketsQueryRelevant but missed
Run a query to see what top-k retrieval finds — and what it misses.
Step three

Worked Example: a Multi-Hop Question

Question: "What changed in the Q3 refund policy and who approved it?" The answer needs a policy chunk from one store and an approval email from another. Watch both strategies attempt it.

Naive RAG

Agentic Search

Step four

Known Limits of Traditional RAG

Chunk boundary loss

Facts that span a chunk boundary are cut in half. Neither half embeds close enough to the question, so the fact is unretrievable even though it is in the corpus.

Top-k ceiling

k is fixed before you know how much evidence the question needs. Multi-part questions need evidence beyond slot k, which is silently dropped.

Stale indexes

Embeddings are computed at ingest time. Documents edited after indexing return outdated chunks with full confidence until a re-index runs.

Multi-hop questions

One-shot similarity cannot follow references: policy points to an approval email points to a ticket. Each hop needs a new, informed query.

Distributed silos

Real corpora live in many indexes with different embedding models and permissions. A single nearest-neighbor search cannot rank across incompatible spaces.

How agents differ

An agent plans, issues targeted searches per store, reads results, follows references, re-queries with new terms, and stops when evidence is sufficient — retrieval becomes a loop, not a lookup.

Step five

Check Your Understanding

Five quick questions. Instant feedback with explanations.