Agentic RAG
Agentic retrieval-augmented generation (RAG) lets a model-directed agent choose how to gather evidence while answering a question. It can revise a query, choose a source or retrieve again after inspecting results, adding decisions to the retrieval process.
A support question asks whether a custom order can be returned. A fixed RAG pipeline retrieves passages and generates an answer. An agentic version can notice that the general policy leaves custom orders unresolved, then search specifically for that exception before answering.
Multiple searches alone do not make the system agentic. Code can issue a fixed set of queries. The defining distinction here is that the agent’s assessment influences what retrieval action happens next. In episode 8 at 6:03, Bob van Luijt discusses agents requesting more information through feedback loops; writing information back is part of his broader example, not a requirement of every agentic RAG design.
This matters when the initial evidence is incomplete, but each additional search adds cost and delay. Bound searches, record which evidence was added and allow an answer that says support is missing. Compare against a fixed retrieval baseline on the same questions. A more elaborate loop is worthwhile only when its measured outcomes justify it.
Hear it from the guest
“So now the model itself says like, Hey, I need more information from Weaviate”
Quotes lightly edited to remove filler words.
Sources
- Anthropic: Building effective agents — Distinguishes model-directed processes from predefined code paths; the retrieval example here applies that distinction.
- Yao et al.: ReAct — Demonstrates model-directed information gathering and decisions based on observations.
Go deeper
- Hugging Face: Agentic RAG course course
Build retrieval as an agent tool and compare model-directed searches with a fixed retrieval pipeline.
