Vector Database

A vector database is a store designed for one job: hold millions of embeddings and, given a query vector, return the closest matches in milliseconds. Regular databases are built to match exact values; a vector database is built to match by similarity, which is what semantic search and retrieval need.

In a RAG system, the vector database is the retrieval layer. Your documents go in as embeddings; at query time the system embeds the question, asks the database for the nearest chunks, and hands those to the model. Picking and tuning one — how it indexes, how it filters, how it scales — is a core infrastructure decision, because retrieval quality caps how good the final answer can be.

Filtering is one place scale shows up. Many queries want the nearest vectors that also match a condition, such as a product category, a date range, or a customer. In episode 8, Weaviate’s Bob van Luijt describes filtered search that customers didn’t notice at a million products turning significantly slower at a billion, which is why Weaviate added a dedicated method for filtering in vector search.

See AI vocabulary in plain English for this term alongside the other language of models, data, agents, and infrastructure.