RAG & Retrieval

The original RAG paper combined a pretrained sequence-to-sequence model with a dense index of Wikipedia passages and a neural retriever. It compared conditioning on the same passages for a whole answer with selecting different passages per token (Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks). That makes retrieval part of the system to evaluate. For a support assistant, test whether it finds the right policy before judging how well it writes the answer.

Bob van Luijt in episode 8 discusses the evolution of retrieval architecture. Sudhir Hasbe in episode 55 argues that data structure and relationships matter for hallucinations. What is RAG? introduces the pipeline, and how to evaluate a RAG system separates retrieval quality from answer quality. Vector database vs knowledge graph compares two ways to organize the evidence an application can retrieve.