12 Types of Retrieval-Augmented Generation (RAG)

Original RAG models combine a pre-trained language model (parametric memory) with an external knowledge source (non-parametric memory), like a dense vector index of documents such as Wikipedia. RAG retrieves relevant information from this source during generation, enhancing the model's accuracy and specificity. → Retrieval-Augmented Generation for

Knowledge-Intensive NLP Tasks

Modular RAG is an advanced framework that breaks down complex RAG systems into independent modules and specialized components. Unlike traditional RAG's simple "retrieve-then-generate" process, Modular RAG offers flexible and customizable configurations like routing, scheduling, and combining processes. → Modular RAG: Transforming RAG Systems into LEGO-like Reconfigurable Frameworks

LongRAG is an improved version of RAG model, which processes larger text units (4,000 tokens instead of 100 words), reducing the number of units to search through. This "long retriever" and "long reader" approach enhances accuracy and performance in extracting answers from large texts without extra training. → LongRAG: Enhancing Retrieval-Augmented Generation

with Long-context LLMs

Self-RAG (Self-Reflective approach) allows the model to retrieve and reflect on information only when needed. It outperforms other models like ChatGPT in tasks requiring reasoning and fact-checking. → SELF-RAG: LEARNING TO RETRIEVE, GENERATE, AND CRITIQUE THROUGH SELF-REFLECTION

Corrective RAG (CRAG) uses an external retrieval evaluator to refine the quality of retrieved documents. It selectively focuses on key information, enhancing the accuracy and robustness of generated content. → CRAG: Corrective Retrieval Augmented Generation

Speculative RAG combines two types of LMs: a smaller, specialized LM for producing multiple drafts in parallel, and a larger generalist LM that verifies these drafts to find the best answer. It enhances both effectiveness and speed of the system. → SPECULATIVE RAG: ENHANCING RETRIEVAL AUGMENTED GENERATION THROUGH DRAFTING

Graph RAG approach presented by Microsoft organizes data into a graph structure, representing text data and its interrelations. Graph RAG is a valuable addition to RAG systems to handle query-focused summarization at scale. → From Local to Global: A GraphRAG Approach to Query-Focused Summarization

Multi-Head RAG uses different parts of the model’s attention mechanism to capture various aspects of a query, making it easier to find and use relevant information. It improves retrieval accuracy, especially for complex queries. → Multi-Head RAG: Solving Multi-Aspect Problems with LLMs

Golden-Retriever is a RAG model that uses reflection-based question augmentation to handle domain-specific jargon and context in industrial knowledge bases, ensuring the retrieval of the most relevant documents. → Golden-Retriever: High-Fidelity Agentic Retrieval Augmented Generation for Industrial Knowledge Base

Adaptive RAG for conversational systems instead of always retrieving external knowledge, assesses the conversation context and decides if RAG is necessary. This approach improves response quality by only using RAG when beneficial, leading to more accurate and confident answers. → Adaptive Retrieval-Augmented Generation for Conversational Systems

RankRAG is a framework that trains the model to both rank relevant contexts and use it to answer questions. It excels at knowledge-intensive tasks. → RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMs

EfficientRAG efficiently handles multi-hop questions by generating new queries without needing LLMs at each step and filtering out irrelevant information. → EfficientRAG: Efficient Retriever for Multi-Hop Question Answering