Understanding RAG in One Article

RAG (Retrieval Augmented Generation): retrieval-augmented generation

0 Introduction

Before we begin: the author is also a beginner and wrote this article with a learning mindset. If there are any problems, you are welcome to contact me by email with criticism and corrections.

This article systematically introduces the basic concepts, core technical principles, development background, and typical architecture of RAG, then takes a deeper look at frontier directions such as multimodal RAG (retrieval-augmented generation for text-image and text-video, for example) and Graph RAG (retrieval-augmented generation based on graph-structured data), and finally analyzes the current challenges and future development trends.

1 Basic Concepts

RAG (Retrieval Augmented Generation), whose Chinese name is 检索增强生成, is in plain terms a technique that improves the generation quality of large models through retrieval, and it aims to improve the accuracy and credibility of generative AI. It realizes the combination of large language models with external knowledge bases and is one of the key routes for mitigating hallucinations and improving traceability. The concept was first proposed by the Facebook AI Research team in 2020, which published the groundbreaking paper that named RAG. Today, RAG has developed into a vast family of methods and is considered an important direction toward more reliable generative AI.

2 Background and Technical Principles

2.1 Background

Large models can “remember” a large amount of knowledge even without an external knowledge base, but obvious limitations remain: knowledge stored in parameters is expensive to update, and answers often lack verifiable sources of evidence; when encountering obscure knowledge, the latest information, or long-chain reasoning, the error rate rises; in multi-turn dialogue and long-context scenarios, problems such as information forgetting, topic drift, and evidence confusion can also occur.

For example, you ask who aorangehc is, but it starts talking about who Einstein is. This kind of generation that is irrelevant to the question or factually wrong is usually called hallucination (LLM Hallucination). Hallucination is not an occasional failure of some particular model, but a systematic phenomenon that probabilistic generation tends to exhibit when information is insufficient, the objective is unclear, or evidence is missing. Researchers therefore began to introduce constraints and external information from outside the model to reduce the occurrence of such problems.

One approach is to perform instruction fine-tuning and alignment training on the model after pretraining, so that when it is uncertain it tends to answer “I don’t know” or to ask for clarification. Another approach is for users to use prompts to constrain the answer scope, evidence requirements, and output format, reducing the room for the model to improvise.

Yet another is to introduce an external knowledge base. When we use large-model applications such as ChatGPT, Gemini, Kimi, Doubao, and DeepSeek, we often see web search or knowledge-base Q&A features: the system searches external data sources for web pages or document snippets based on the question, provides the retrieval results to the model as evidence, and the model then organizes an answer based on the evidence and its own language ability, thereby significantly improving factuality and explainability. This process of “retrieve first, then generate” is the typical paradigm of RAG.

2.2 Technical Principles

The large models in today’s sense have many capabilities. In daily life we mostly use large models to ask questions and have them solve the problems we encounter—for example, when Python code throws an error, we throw the error message at the model and it gives suggestions for fixes; the main capability used in this process is instruction following. In addition, there are in-context learning, reasoning, tool use, and so on, and as technology develops, the capabilities that large models can call upon continue to expand.

In RAG, the main capability used is the in-context learning ability of large models: relevant knowledge is obtained through retrieval and provided to the model as context, and the model then summarizes based on the question and the evidence and generates an answer.

The core idea of RAG is to introduce a retrieval module into the generation task. After the model receives input, the retrieval module first retrieves relevant material from an external knowledge base (such as Wikipedia, proprietary document collections, or enterprise internal materials), and then the retrieved content is fed into the generative model together with the original input to generate the final answer or text. This “retrieve-generate” strategy enables the model to look up and use the knowledge it needs on the spot, instead of relying entirely on old knowledge stored in its parameters. OpenAI’s WebGPT work is a typical case: it lets GPT-3 search web pages in a controlled browsing environment, extract cited snippets, and answer long questions on that basis, thereby improving the factuality and verifiability of answers. Introducing retrieval not only reduces the incidence of hallucination but also allows the knowledge base to be expanded and updated more flexibly—for example, users can connect new data sources for the system to use as needed.

Since RAG was proposed, major research teams have rolled out improved retrieval-augmented models one after another. For example, DeepMind’s RETRO model, released in 2021, performs retrieval augmentation by connecting to an ultra-large-scale text database, and at a smaller parameter scale achieved results comparable to the then-largest models, and even better on some benchmarks, demonstrating the potential of retrieval for “obtaining stronger knowledge capability with a smaller model.” Another example is Meta’s Atlas model, introduced in 2022, which applies a pretrained retrieval-augmented language model to few-shot learning and, on knowledge-intensive tasks such as open-domain question answering, achieved performance competitive with very large models using a significantly smaller parameter scale. These results show that retrieval augmentation can achieve, or even approach, the effect of very large models at a smaller model scale, highlighting the efficiency advantages and engineering value of combining parameters with retrieval.

3 RAG Framework

Image cited from Retrieval-Augmented Generation for AI-Generated Content: A Survey

A typical RAG system contains two core components: a retriever and a generator.

3.1 Basic RAG Pipeline

First, the input query is encoded by the retrieval module into a vector representation, and a similarity search is performed in the vector index of the external knowledge base to retrieve several relevant text snippets; re-ranking techniques may then be applied to improve precision. Subsequently, these retrieved evidence snippets are provided to the generative model together with the original query, and the model generates the final answer based on the question and the evidence. By incorporating external knowledge into the generation process, RAG can make answers closer to facts and, in engineering terms, support giving citation sources and evidence snippets, thereby improving explainability and auditability.

Classic RAG often uses a fixed set of retrieved documents when generating the entire answer; there are also more advanced variants such as RAG-Token, which can dynamically use different retrieved evidence at different positions during generation, thereby improving the flexibility of information use.

Either way, the fusion of retrieval results and the generative model is key. Many implementations concatenate the retrieved text directly into the model input, while some research also explores using retrieval information inside the model in a more structured way, so that the model can more stably align with evidence during generation and reduce irrelevant digressions. In practice, the retriever and generator can be optimized independently in stages, or, under specific settings, jointly trained or end-to-end fine-tuned, so that the retriever is more inclined to return the evidence most helpful for generation and the generator is better at citing and integrating evidence.

3.2 Retriever

Retrieval refers to identifying and obtaining relevant information according to an information need. Specifically, we can view an information resource as a key-value store, where each key corresponds to a value, and the key and value may be the same. Given a query, the goal is to use a similarity function to search for the top-k most similar keys and obtain the values paired with them. Depending on the similarity function, existing retrieval methods can be divided into sparse retrieval, dense retrieval, and other types.

3.2.1 Sparse Retriever

Sparse retrievers use term-matching metrics such as TF-IDF and BM25 and are mainly used for document relevance retrieval. These methods achieve efficient search by analyzing word statistics in the text and building an inverted index. In essence, BM25 is a strong baseline in large-scale web search; it integrates inverse document frequency weights, the number of occurrences of query terms, and other related metrics.

3.2.2 Dense Retriever

Dense retrieval represents queries and keys with dense embedding vectors and builds approximate nearest neighbor indexes to speed up search; it can be used for retrieval across all modalities. Dense retrieval encodes the original data into vectors through a pretrained model, and then retrieves by computing similarity scores; similarity can be computed by cosine, inner product, L2 distance, and so on.

To improve performance, RAG systems today generally adopt a hybrid retrieval mode of sparse retrieval plus dense retrieval to improve results.

3.3 Generator

The generator combines the retrieved content to generate the required content.

With the development of the AIGC era, generators have also continued to develop, enrich, and mature, and different generator models can be used for different scenarios, for example:

• Transformer models for text-to-text tasks

• VisualGPT for image-to-text tasks

• Stable Diffusion for text-to-image tasks

• Codex for text-to-code tasks

4 RAG Enhancements

Yes—the enhancement in retrieval-augmented generation is also a very important research topic, with many techniques.

4.1 Input Enhancement

The content initially fed into the retriever has a significant impact on the final result of the retrieval stage. The main ways of input enhancement are as follows.

4.1.1 Query Transformation

Query transformation can enhance retrieval results by modifying the input query.

Query2doc and HyDE use the original query to generate a pseudo-document, which is then used as the query for retrieval. The pseudo-document contains richer relevant information, which helps retrieve more accurate results.

TOC uses the retrieved content to decompose an ambiguous query into multiple clear sub-queries; these sub-queries are sent to the generator and, after aggregation, produce the final result.

RQ-RAG decomposes complex or ambiguous queries into clear sub-queries for fine-grained retrieval and synthesizes the responses, thereby providing a coherent answer to the original query.

4.1.2 Data Augmentation

Data augmentation means improving the data before retrieval, including techniques such as removing irrelevant information, disambiguating, updating outdated documents, and synthesizing new data.

Make-An-Audio uses caption generation and audio-text retrieval to generate captions for non-speech audio, alleviating the data sparsity problem, and adds random concept audio to improve the original audio.

LESS optimizes downstream task dataset selection by analyzing gradient information, aiming to improve the model’s response performance to instruction prompts.

ReACC pretrains a code retrieval model through renaming and dead code insertion.

4.2 Retriever Enhancement

4.2.1 Recursive Retrieval

Recursive retrieval means performing multiple searches to obtain richer, higher-quality content.

ReACT uses chain-of-thought (CoT) to decompose the query for recursive retrieval and provide richer information.

RATP uses Monte Carlo tree search for simulation to select the optimal retrieval content, which is then templated and forwarded to the generator for output.

4.2.2 Chunk Optimization

Chunk optimization refers to adjusting chunk size to improve retrieval results.

LlamaIndex integrates a series of chunk optimization methods, one of which follows a “small-to-big” principle. Its core idea is to pinpoint finer-grained content while returning richer information. For example, sentence window retrieval obtains small text chunks and returns the relevant sentence window around the retrieved snippet. In auto-merging retrieval, documents are organized into a tree structure. The process first obtains child nodes, then retrieves the parent node that contains the content of those child nodes.

RAPTOR uses recursive embedding, clustering, and text chunk summarization until no further clustering is possible, thereby building a multi-level tree structure.

Prompt-RAG improves retrieval accuracy by pre-generating a table of contents, enabling the model to autonomously select relevant chapters based on the query.

4.2.3 Retriever Fine-tuning

In dense retrieval, the retriever relies on an efficient embedding model to represent relevant content and provide input to the generator, thereby improving system performance.

Fine-tuning can be performed with domain- or task-specific data to improve performance in the target domain.

REPLUG treats the language model as a black box and updates the retrieval model based on the final results.

APICoder fine-tunes the retrieval model using Python files along with API names, signatures, and descriptions.

EDITSUM fine-tunes the retrieval model to reduce the Jaccard distance between summaries after retrieval.

The latest Qwen3 series has open-sourced a variety of embedding and rerank models and released a technical report, with very good results. Among them, the smaller models are suitable for fine-tuning tests and performance comparisons.

4.2.4 Hybrid Retrieval

Hybrid retrieval means using multiple different retrieval methods at the same time or extracting information from multiple different sources.

RAP-Gen, BlendedRAG, and ReACC all use both dense and sparse retrievers to improve retrieval quality.

Rencos uses a sparse retriever to retrieve similar code snippets at the syntactic level and a dense retriever to retrieve similar code snippets at the semantic level.

BASHEXPLAINER first uses a dense retriever to capture semantic information, then uses a sparse retriever to obtain lexical information.

RetDream first retrieves through text, then retrieves through image embeddings.

CRAG’s distinctive feature is a retrieval evaluator that measures the relevance of documents to the query and generates three kinds of retrieval responses according to confidence: if the results are accurate, they are used directly for knowledge refinement; if the results are incorrect, a web search is performed; and for ambiguous cases, a hybrid approach is adopted.

4.2.5 Rerank

Reranking is probably the most common approach: it reorders the retrieved content to obtain greater diversity and better results. Many reranking models are also open-source and ready to use.

Re2G applies a reranker model after the traditional retriever to reduce the impact of information loss caused by compressing text into vectors.

AceCoder uses a selector to rerank the retrieved programs, reducing redundant programs and obtaining diverse retrieved programs.

XRICL uses a distillation-based example reranker after retrieval.

Rangan adopts a quantitative influence measure, measuring the similarity of data subsets by evaluating the statistical deviation between queries and references, and reranks the retrieval results.

UDAPDR uses a large language model to cost-effectively generate synthetic queries for training domain-specific rerankers, then applies multi-teacher knowledge distillation to develop a collaborative retriever.

4.2.6 Retrieval Transformation

Retrieval transformation improves output results by reformulating the retrieved content to better stimulate the generator’s potential.

FILCO can efficiently remove irrelevant content from retrieved text and retain only relevant supporting content, thereby simplifying the generator’s task and helping to accurately predict answers.

FiD-Light first uses an encoder to convert the retrieved content into vectors, then compresses them, greatly reducing latency.

4.3 Generator Enhancement

4.3.1 Prompt Engineering

Prompt engineering is a technique focused on improving the output quality of large language models, such as prompt compression, step-back prompting, active prompting, and chain-of-thought prompting, all of which are applicable to the large language model generators in retrieval-augmented generation systems.

LLMLingua applies a small model to compress the overall length of the query in order to accelerate model inference, mitigate the negative impact of irrelevant information on the model, and alleviate the lost-in-the-middle phenomenon.

ReMoDiffuse uses ChatGPT to decompose complex descriptions into structured text scripts.

ASAP incorporates example tuples containing input code, function definitions, analysis results, and corresponding comments into the prompt to obtain better results.

CEDAR uses designed prompt templates to organize code demonstrations, queries, and natural language instructions into prompts.

XRICL uses chain-of-thought techniques to add translation pairs as intermediate steps in cross-lingual semantic parsing and reasoning.

ACTIVERAG adopts a cognitive connection mechanism to calibrate the large language model’s intrinsic cognition and applies chain-of-thought prompts in answer generation.

Make-An-Audio can use other modalities as input, which can provide richer information for subsequent processes.

4.3.2 Decoding Tuning

Decoding tuning includes enhancing generator control by fine-tuning hyperparameters to improve diversity, limit the output vocabulary, and make other adjustments.

InferFix balances the diversity and quality of results by adjusting the decoder’s temperature. (Temperature can be understood as a parameter that controls the degree of divergence in the content generated by the model.)

SYNCHROMESH limits the decoder’s output vocabulary by implementing a completion engine to eliminate implementation errors.

4.3.3 Generator Fine-tuning

Fine-tuning the generator can enhance the model’s ability to master more precise domain knowledge, or make it better match the retriever.

RETRO fixes the retriever’s parameters and uses a chunked cross-attention mechanism in the generator to combine the query and the retriever’s content.

CARE uses image, audio, and video-text pairs to train the encoder, then fine-tunes the decoder (generator) to simultaneously reduce caption and concept detection losses while keeping the encoder and retriever fixed.

Animate-A-Story uses image data to optimize the video generator, then fine-tunes LoRA adapters to capture the appearance details of a given character.

4.4 Result Enhancement

4.4.1 Output Rewriting

Output rewriting refers to rewriting the content generated by the generator in specific scenarios to meet the needs of downstream tasks.

SARGAM optimizes output in code-related tasks by employing a special Transformer that combines a deletion classifier, a placeholder classifier, and an insertion classifier, making it better fit real-world code contexts.

Ring reranks candidate results according to the average log probability of each token produced by the generator, thereby obtaining diverse results.

CBR-KBQA corrects results by aligning the generated relations with the relations presented in the local neighborhood of the query entity in the knowledge graph.

4.5 Pipeline Enhancement

RAG pipeline enhancement refers to optimizing the entire RAG pipeline to obtain better performance results.

4.5.1 Adaptive Retrieval

Some RAG research shows that retrieval does not always improve the final result. When the model’s inherent parametric knowledge is sufficient to answer the relevant question, excessive retrieval may lead to wasted resources and may cause confusion. Therefore, some methods study whether RAG should be performed.

• Rule-based

  • FLARE actively decides during generation whether and when to perform retrieval based on probabilities.

  • Efficient-KNNLM combines the generation probabilities of KNN-LM and NPM with the hyperparameter λ to determine the ratio of generation to retrieval.

  • There is also statistical analysis of questions to provide direct answers for high-frequency questions and apply retrieval-augmented generation to low-frequency questions, etc.

• Model-based

  • Self-RAG uses a trained generator to decide whether to perform retrieval based on retrieval tokens under different user queries.

  • SKR uses the large language model’s own ability to pre-judge whether it can answer the question; if it can, no retrieval is performed.

  • Rowen translates a question into multiple languages and checks the consistency of the answers in those language versions to decide whether information retrieval is needed.

  • AdaptiveRAG uses a smaller language model as a classifier to dynamically decide whether to retrieve based on the complexity of the query.

4.5.2 Iterative Retrieval-Augmented Generation

Iterative retrieval-augmented generation gradually optimizes results by repeatedly looping through the retrieval and generation stages—in practice, performing multiple rounds of operations.

RepoCoder adopts an iterative retrieve-generate approach for code completion, using previously generated code to refine the query, so as to better utilize scattered information and improve results.

ITER-RETGEN uses the generator’s output to identify knowledge gaps, retrieve the necessary information, and inform future generation cycles, thereby iteratively improving content quality.

SelfMemory iteratively uses the retrieval-augmented generator to build a large memory pool, from which a memory selector picks outputs to inform the next generation cycle.

RAT first has the large language model generate content through zero-shot chain-of-thought prompting, then revises each thought step by retrieving knowledge from an external knowledge base.

5 RAG’s Multi-hop, Multi-domain Problem (Graph RAG)

In RAG there is a very classic bottleneck problem: some questions are not about finding a certain most similar passage, but about stringing together clues scattered across multiple documents, multiple concepts, and even multiple domains to form a comprehensive conclusion. When an answer requires traveling along shared entities, attributes, or relations between different pieces of information and generating new insights on that basis, relying on vector similarity retrieval alone is often insufficient. This is the multi-hop and multi-domain problem that RAG has to face.

GraphRAG (Graph Retrieval-Augmented Generation) can be seen as an upgrade path prepared for these two types of problems: it does not merely retrieve semantically similar text chunks, but explicitly represents knowledge as a graph structure—for example, extracting entities and relations from text to form a knowledge graph or graph index—and uses the relational structure in the graph to organize evidence and advance reasoning.

IBM’s definition of GraphRAG is very close to engineering practice: GraphRAG is an advanced form of RAG that introduces graph-structured data; unlike relying solely on vector retrieval of semantically similar text, it uses the relational structure of the graph to retrieve and process information, and is especially good at multi-hop reasoning, relational context, and the organization and querying of hierarchically structured data.

6 Multimodal RAG

6.1 Multimodal Large Models

With the continuous development of large language model technology, multimodal and even omni-modal large models have become important research subjects and continue to develop. Multimodal large models are now gradually becoming mainstream and bringing convenience to people’s lives. It is worth mentioning that Gemini’s image generation is really strong—the image below was generated by it, and it intuitively shows the complete technology evolution roadmap of multimodal large models, starting from the early basic image-text connection (such as CLIP and DALL-E), going through multi-sensory fusion (such as GPT-4V and Gemini), and finally moving toward omni-modal real-time interaction and artificial general intelligence (AGI). The prompt used was: “Generate an image introducing the development history of multimodal large models”. A very simple prompt, but the generated image is quite sci-fi.

6.2 Multimodal Retrieval-Augmented Generation

With the development of multimodal large models, researchers have begun to explore multimodal retrieval-augmented generation (Multimodal RAG), that is, enabling the system not only to retrieve text but also to use multiple data sources such as images, audio, video, and code as knowledge evidence. What is rather special is that the core idea and pipeline of RAG remain basically consistent across modalities, but the retriever, generator, and related technical details need to be adjusted.

The basic idea of multimodal RAG is: for data of different modalities, introduce corresponding encoders to map them into a comparable vector space, thereby supporting cross-modal retrieval. For example, the system can encode images into vectors with a vision model and text into vectors with a language model, and build a unified or alignable retrieval index. When a user poses a query in text, the retrieval module can return not only relevant text snippets but also relevant images, audio clips, or video clips as supplementary information. At present, the most common form of multimodal retrieval involves the two modalities of image and text; deeper details are often related to VLMs and will be introduced further later.

Multimodal RAG shows unique value in tasks that require integrating visual and textual information. For example, in cross-media question answering, when a user provides an image and asks a related question, the system can first understand the image content and then combine it with retrieved external knowledge text to answer deeper questions, such as the background of the person in the image, location information, or the context of historical events. Another example is video question answering or multimedia search, where the system can comprehensively retrieve subtitle text, video key frames, clip summaries, and even audio cues to obtain more complete information.

Some studies apply multimodal RAG to visual question answering tasks, demonstrating that combining image content with external knowledge can answer some questions that are difficult to answer from the image alone. Open-source tools and frameworks supporting multimodal retrieval have also emerged in the industry, for example using CLIP-like models to encode image-text for cross-modal search. Overall, multimodal RAG expands the limitation of traditional RAG to handling only plain text, providing the system with an external evidence channel oriented toward the multimodal world, making generation results richer and more verifiable.

6.3 Challenges of Multimodal RAG

1. Data formats differ across modalities, and how to align multimodal representations is key. A common approach is contrastive learning based on large-scale image-text pairs, audio and transcribed text, and so on, so that vectors from different modalities fall into a semantically consistent space.

2. The retrieval process may need to jump between modalities, for example searching text with an image or images with text, which requires the index to store multimodal information and support queries from any modality to content in any modality.

3. Multimodal data is large in scale and retrieval computation is expensive, especially when performing approximate nearest neighbor retrieval over massive images or video clips, where both cost and latency are challenges. Some systems adopt a strategy of converting to text before retrieval, for example performing OCR on images or generating descriptions before entering the text retrieval pipeline, to be compatible with existing text retrieval systems, but this may lose key visual details. More cutting-edge methods tend to preserve the original modality information, let retrieval results directly return image or video content, and have a multimodal generative model complete understanding and generation end-to-end, thereby reducing the information loss caused by intermediate representations.

7 Applications of RAG

7.1 Application Categories

• Text RAG

  • Question answering: responding to questions by extracting information from large and comprehensive collections of text resources.

  • Fact checking: usually refers to judging whether a given natural language text and the related claims or assertions are consistent with the facts in the text.

  • Commonsense reasoning: refers to the ability of machines to use acquired external knowledge and its application to infer or make decisions about problems or tasks in a human-like way.

  • Human-machine dialogue: includes the machine’s ability to understand natural language and the ability to skillfully use this skill for seamless communication with humans.

  • Neural machine translation: automatically translating text from a source language into a target language.

  • Event extraction: this is a process in natural language processing that involves identifying and classifying specific events in text, and associating these events with related entities.

  • Summarization: extracting key information from lengthy text and generating a concise, coherent summary that captures the main themes.

• Code RAG

  • Code generation: converting natural language descriptions into code implementations.

  • Code summarization: converting code into natural language descriptions.

  • Code completion: code completion is similar to the code version of the next-sentence prediction task.

  • Automated program repair: query-based RAG is commonly used in automated program repair to help generative models fix defective code.

  • Text2SQL and code-based semantic parsing: semantic parsing converts natural language into clear structured representations, such as SQL or other domain-specific languages, usually with the help of code.

• Knowledge RAG

  • Knowledge base question answering: usually uses a knowledge base to determine the correct answer to a question.

  • Knowledge-enhanced open-domain question answering: mainly the application of structured knowledge.

  • Table question answering: tables, as another form of structured knowledge, also help question answering.

• 3D RAG

  • Text-2-3D: retrieval can be used to enhance 3D asset generation.

• Image RAG

  • Image generation: refers to the process of creating new images, usually using algorithms from the fields of artificial intelligence and machine learning.

  • Image summarization: the process of generating textual descriptions of images.

• Video RAG

  • Video summarization: converting visual content into descriptive statements.

  • Video question answering and dialogue: video QA and dialogue generate single-turn or multi-turn responses consistent with the video content.

• Audio RAG

  • Audio captioning: audio captioning is essentially a sequence-to-sequence task that generates natural language data for audio data.

  • Audio generation: usually synthesizes audio based on natural language prompts.

• Scientific Domain RAG

  • Drug discovery: the goal of drug discovery is to generate molecules that simultaneously satisfy multiple properties.

  • Biomedical informatics enhancement: by retrieving information from domain-specific biomedical databases, the expressive power of large language models is improved, thereby enhancing the model’s ability to provide valuable guidance for tasks in the medical field.

  • Mathematical question answering: retrieval-augmented generation in the mathematical field simplifies the problem-solving process, promotes research innovation, and optimizes educational strategies.

7.2 Application Domains

Personal and enterprise internal knowledge bases: such applications are dedicated to solving the retrieval and question-answering challenges of private data. They structure and vector-index a company’s internal documents, manuals, or personal notes; when a user asks a question, the system can locate the relevant paragraphs in the files and have a large model generate an answer based on the evidence, thereby replacing traditional keyword search in many scenarios and significantly improving the efficiency of information acquisition.

Deep reading and provenance of academic papers: for scenarios where researchers need to handle massive amounts of literature and have extremely high accuracy requirements, such applications can retrieve evidence across multiple papers and generate reviews or answers. Their core characteristic is an emphasis on evidence traceability and citation alignment, making every key conclusion trace back to the original source as much as possible, thereby minimizing the risk brought by hallucination.

Intelligent assistants for ultra-large-scale codebases: to solve the problem that traditional large models find it difficult to read a complete project’s code in one go, such applications index the structure, symbols, and dependencies of code repositories, allowing developers to query project architecture or specific function implementation details in natural language, and provide explanations and modification suggestions within the scope of the evidence, thereby understanding and maintaining complex legacy code more efficiently.

7.3 Ima

ima is an AI knowledge base tool developed by Tencent; its core is RAG, and it currently supports importing WeChat Official Account articles to build a knowledge base for retrieval.

In a WeChat Official Account article, click Forward to import the article into the knowledge base, and then you can ask related questions through AI.

7.4 Coze

Coze Space is ByteDance’s AI application development platform Coze, a multi-agent collaborative development environment that allows developers to orchestrate multiple AI Agents, tools, data sources, and workflows into a complex automated task system within a unified canvas-style interface, enabling the construction of advanced AI applications ranging from a single chatbot to multi-role collaboration and multi-step execution.

Below, we use its Agent and knowledge base construction features to build a Bagu Q&A assistant.

First, import text to build a knowledge base, then create an agent and test it. The test verifies that it can retrieve the corresponding information from the knowledge base; this is a knowledge-base RAG application.

7.5 Ragflow

RAGFlow is a leading open-source retrieval-augmented generation (RAG) engine that combines cutting-edge RAG with agent capabilities to create an excellent context layer for large language models. It provides a streamlined RAG workflow suitable for enterprises of any scale. Supported by a fused context engine and prebuilt agent templates, RAGFlow enables developers to turn complex data into high-fidelity, production-ready AI systems, with exceptional efficiency and precision.

It can be deployed and used locally.

8 Datasets&Benchmarks

8.1 Others\x20;

• Noise robustness: testing whether large language models can extract necessary information from noisy documents

• Negative rejection: evaluating whether large language models can refuse to answer when the retrieved content is insufficient

• Information integration: checking whether large language models can acquire knowledge and answer by integrating multiple retrieved contents

• Counterfactual robustness: determining whether large language models can identify counterfactual errors in retrieved content

• Faithfulness: evaluating factual accuracy based on retrieved content

• Answer relevance: determining whether the result addresses the query

• Context relevance: evaluating the relevance and conciseness of the retrieved content

• CRUD-RAG divides RAG tasks into four types: create, read, update, and delete, and evaluates them through text continuation, question answering, hallucination correction, and open-domain multi-document summarization

• A dataset I used before, TreezzZ/WebQA, for testing multimodal performance; this thing is itself a benchmark containing question-answer pairs, and it can be used after processing with a simple script

8.2 Multimodal RAG

8.2.1 Datasets

8.2.2 Benchmark

8.2.3 Evaluation Metrics

• Retrieval evaluation

  • Accuracy

  • Recall and Recall@K: used to check relevant items among the top K results; preferred over standard recall

  • Precision and Top-K accuracy

  

  • F1 score

  • Mean reciprocal rank (MRR): used to measure the rank position of the first relevant result in the returned list, where Q is the total number of queries. r⁢a⁢n⁢kq is the rank of the first relevant result for query q.

  

• Modality evaluation

  • Text evaluation

    • Exact match (EM)

    • BLEU

    

    

    • ROUGE: used to evaluate text summarization and generation

    • ROUGE-N: used to measure the overlap of N-grams between generated text and reference text

    

    • ROUGE-L: measures the longest common subsequence (LCS) between generated text and reference text

    

    • METEOR

  • Image evaluation

    • Image captioning

      • CIDEr (Consensus-based Image Description Evaluation): uses TF-IDF and cosine similarity to measure caption quality

      • SPICE (Semantic Propositional Image Caption Evaluation)

      • SPIDEr: combines CIDEr and SPICE

    • Semantic alignment

      • BERTScore

      • CLIPScore

      

    • Image quality

      • FID

      

      • KID

      • Inception Score: evaluates the diversity and quality of images through classification probabilities

9. From Concept to a Small Project

Many structures and metrics have been listed above, but when actually getting started, you can first narrow the problem down. Take a few documents you have read, build a question-answering tool that can trace the source of answers, then gradually switch retrieval methods.

9.1. Get the Minimal Pipeline Working First

Give documents and snippets stable IDs, and record their source and page numbers; retrieval results should return both text and location. The context passed to the model should contain only the snippets found in this round, and the answer should be required to carry source numbers.

For example, below is a small custom test, not the actual configuration of any existing project:

文档 A：订单接口的请求超时为 3 秒。
文档 B：订单号不能为空，同一订单号不能重复创建。

问题：订单接口多长时间超时？
期望：3 秒，并引用文档 A。

问题：退款接口的超时是多少？
期望：当前资料没有说明，而不是沿用订单接口的数字。

This example checks two things at once: whether evidence can be found when there is an answer, and whether guessing can stop when there is no answer. Retrieved document content is used as reference material; just because a snippet says “ignore other requirements” does not mean it should be treated as a system instruction.

9.2. Locate the Problem Layer First

• The correct snippet was not retrieved: check document parsing, chunking, query formulation, and ranking.
• It was found, but the context was truncated: check snippet length, concatenation order, and context budget.
• The context contains the answer, but the response is wrong: check the generation prompt, whether citations correspond, and whether the model went beyond the evidence.
• After updating a document, it still answers with old content: check index updates and cache invalidation.

The fixes for these problems differ. First preserve the query, retrieved snippets, final context, and answer, then decide whether to change the model or the retrieval.

9.3. Build a Repeatable Set of Checks

For each test question, record “whether an answer exists, the relevant snippet IDs, and the acceptable key points of the answer.” Include synonymous phrasings, exact IDs, questions with no answer, and mutually conflicting documents; do not test only one pretty demo.

At the retrieval layer, check whether the correct snippet can enter the top K results; at the generation layer, check whether the answer is supported by the snippets, whether citations are genuine, and whether the model fabricates when it should refuse to answer. Text similarity metrics such as BLEU and ROUGE can help compare expressions, but they cannot on their own prove that an answer is grounded.

The basic research entry point adds the original RAG paper. The actual BM25 project and startup process are documented in the BM25 Q&A assistant notes. What is added here is the validation method; no experimental results are added that were not actually run.

References

Why everyone uses RAG: this article makes it clear even to beginners

Retrieval-Augmented Generation for AI-Generated Content: A Survey

Ask in Any Modality: A Comprehensive Survey on Multimodal Retrieval-Augmented Generation