RAG: Definition and Examples

RAG, or Retrieval-Augmented Generation, is an architecture that combines two key steps: retrieving relevant information from a knowledge base, then generating a response by an LLM using that retrieved information. This approach overcomes fundamental limitations of language models, especially their knowledge cutoff date and tendency to hallucinate.

Concretely, when a user asks a question, the RAG system first converts the question into a vector (embedding), then searches for the most similar passages in a vector database. These relevant passages are then injected into the prompt sent to the LLM, which can then formulate a response based on factual and up-to-date data.

RAG has become one of the most popular architectures in applied AI because it offers an excellent compromise between performance and cost. Rather than fine-tuning an entire model on specific data (expensive and rigid), RAG allows updating the knowledge base independently of the model. It is the preferred approach for enterprise chatbots, document assistants, and question-answering systems on specialized corpora.

The quality of a RAG system depends heavily on the relevance of the retrieval phase. Poor document chunking, embeddings ill-suited to the domain, or overly simplistic search strategies can significantly degrade results. That is why advanced techniques like re-ranking, semantic chunking, or multi-step RAG have emerged to improve the accuracy of these systems.