A support assistant can retrieve the current returns policy, put the relevant passage beside a customer’s question, and ask the model to answer from it. The policy can change independently of the model’s weights. The original RAG paper combined a pretrained generator with a dense retriever over an external document index, then fine-tuned the generator and the retriever’s query encoder together.
Retrieval need not mean nearest-neighbor vector search: systems can use keywords, structured queries, graphs, or combinations. What makes the pattern RAG is supplying retrieved information to generation.
Example: a correct document about the wrong product can still produce a wrong answer. Check that the retrieved evidence covers the question, that the model’s claims follow from it, and that the user is allowed to see it. Faithfulness and answer relevance measure different parts of that problem.
Bob van Luijt’s episode 8 discussion at 6:03 explains how retrieval can grow into an agent feedback loop. The RAG evaluation guide helps separate missing evidence from a generation error.
See AI vocabulary in plain English for this term alongside the other language of models, data, agents, and infrastructure.
Sources
- Lewis et al.: Retrieval-Augmented Generation — Fine-tunes a generator and query encoder over a fixed document index. Modern RAG also includes frozen models and other retrieval methods.
Go deeper
- What is RAG, and why do AI systems use it? AI, decoded · What Is RAG (Retrieval-Augmented Generation)
- What is agentic RAG, and how is it different from regular RAG? AI, decoded · Agentic RAG vs. Traditional RAG
- Should you use prompting, RAG, or fine-tuning to customize an AI model? AI, decoded · Fine-Tuning vs. RAG vs. Prompting: How to Choose
- Does AI replace human translators? AI, decoded · AI Translation: Where Humans Still Sit in the Loop
From the conversation
-
AI Infrastructure & the Evolution of RAG | Weaviate's Bob van Luijt -
Practical Lessons for GenAI Evals | Chip Huyen & Vivienne Zhang -
How Intercom Cut $250K/Month by Ditching GPT for Open Models | Fergal Reid -
The Enterprise AI Deployment Playbook | ServiceTitan, Indeed & Twilio -
GenAI Predictions for 2025 | Databricks & Cohere -
Context Poisoning is Killing Your AI Agents: How to Stop it | Michel Tricot, Airbyte -
Hallucinations Are a Data Architecture Problem | Sudhir Hasbe, Neo4j -
The AI Framework Era Is Over: Why Context Is the Moat | Jerry Liu of LlamaIndex