.webp)
By now, RAG is an accepted and well-established standard for addressing data relevance in in-context learning. But there are growing concerns around model behaviour when inaccurate data is retrieved. Corrective RAG, also known as Corrective Retrieval-Augmented Generation (CRAG), directly addresses this problem by adding a retrieval evaluator and corrective loop on top of standard RAG. In this article, we’ll break down how CRAG works, why it improves robustness, and where it fits compared to approaches like Self-RAG, including a short Self-RAG vs Corrective RAG comparison.
What is Corrective Retrieval-Augmented Generation (CRAG)?
There are a number of developments taking place in terms of RAG.
The one exciting development is that of Agentic RAG. Agentic RAG is where a hierarchy of agents are use to perform RAG; this method allows for a multi-layered approach where searches are performed across document sources. Various sources are synthesised into one succinct and concise response. There has also been numerous studies published considering the triage of retrieved information together with in-context learning. Corrective Retrieval Augmented Generation (CRAG) is proposed to enhance the robustness of generation when errors in retrieval are introduced. Part of CRAG, is a lightweight retrieval evaluator which assesses the overall quality of retrieved documents, providing a confidence degree to trigger different knowledge retrieval actions.
Also, to address limitations in static and limited corpora, large-scale web searches are used to augment retrieval results. CRAG employs a decompose-then-recompose algorithm for retrieved documents, allowing selective focus on key information and filtering out irrelevant details. CRAG is designed to be plug-and-play and seamlessly integrate with various RAG-based approaches. Experiments conducted during the study using four datasets covering short- and long-form generation tasks show that CRAG significantly enhances the performance of RAG-based approaches.
CRAG overview
Considering the diagram below, a retrieval evaluator is constructed to evaluate the relevance of the retrieved documents to the input. An estimation of the degree of confidence is made based on which different knowledge retrieval actions of Correct, Incorrect, Ambiguous can be triggered.
The proposed method is named Corrective Retrieval-Augmented Generation (CRAG), and it is aimed to self-correct retriever results and enhance document utilisation for generation. A lightweight retrieval evaluator is introduced to assess the overall quality of retrieved documents for a given query. This evaluator is a crucial component in Retrieval-Augmented Generation (RAG), contributing to informative generation by reviewing and evaluating the relevance and reliability of retrieved documents. The retrieval evaluator quantifies a confidence degree, enabling different knowledge retrieval actions such as Correct, Incorrect, Ambiguous based on the assessment. For Incorrect and Ambiguous cases, large-scale web searches are integrated strategically to address limitations in static and limited corpora, aiming to provide a broader and more diverse set of information.
Lastly…a decompose-then-recompose algorithm is implemented throughout the retrieval and utilisation process. This algorithm helps eliminate redundant contexts in retrieved documents that are unhelpful for RAG, refining the information extraction process and optimising the inclusion of key insights while minimising non-essential elements.
Retrieval evaluator
From the diagram above, it is clear that the accuracy of the retrieval evaluator plays a significant role in determining the performance of the entire system.
This algorithm ensures the refinement of retrieved information, optimising the extraction of key insights and minimising the inclusion of non-essential elements, thereby enhancing the utilisation of retrieved data.
Self-RAG vs Corrective RAG: what’s the difference?
Both Self-RAG and Corrective RAG aim to make RAG systems more trustworthy, but they intervene at different stages of the pipeline. Understanding Self-RAG vs Corrective RAG helps clarify when to use each, and why they can be combined.
- Self-RAG focuses on reflective generation. The model critiques or verifies its own outputs and supporting evidence during or after generation. It may re-query, revise, or down-weight certain documents based on internal reflection steps.
- Corrective RAG (CRAG) focuses on corrective retrieval. It evaluates the quality of retrieved documents before generation, classifies them as Correct / Incorrect / Ambiguous, and then adjusts the retrieval strategy accordingly (e.g., web search, expanded retrieval, or more focused queries).
In simple terms, Self-RAG improves how the model reasons over the provided evidence, while Corrective RAG improves the quality and robustness of the evidence itself. In practice, the two are complementary: CRAG can clean and correct retrieval inputs, and Self-RAG can then further reflect on and refine the final answer, delivering a more reliable end-to-end RAG system.
Conclusion
This study focus on challenges faced by Retrieval-Augmented Generation (RAG) approaches when retrieval results are inaccurate, leading to the generation of incorrect knowledge by language models. The proposed solution is something the researches named Corrective Retrieval Augmented Generation (CRAG). The research introduces CRAG as a plug-and-play solution to enhance the robustness of generation by mitigating issues arising from inaccurate retrieval. CRAG incorporates a retrieval evaluator designed to estimate and trigger three different actions. The inclusion of web search and optimised knowledge utilisation operations in CRAG improves automatic self-correction and enhances the efficient utilisation of retrieved documents. Experiments combining CRAG with standard RAG and Self-RAG showcase its adaptability to various RAG-based approaches. The study, conducted on four datasets, illustrates CRAG’s generalisability across both short- and long-form generation tasks.
Previously Published on Medium.
