RARE: Retrieval-Augmented Reasoning Enhancement for Large Language Models

TL;DR

RARE enhances large language models' reasoning and factual accuracy by integrating retrieval-augmented actions within a Monte Carlo Tree Search framework, significantly improving performance on complex knowledge-intensive tasks.

Contribution

Introduces RARE, a novel retrieval-augmented reasoning framework with innovative search actions and a factuality scorer, improving LLM reasoning accuracy and factual integrity.

Findings

RARE enables open-source LLMs to match top models like GPT-4.

Improves reasoning accuracy on commonsense and medical tasks.

Enhances factual consistency in generated responses.

Abstract

This work introduces RARE (Retrieval-Augmented Reasoning Enhancement), a versatile extension to the mutual reasoning framework (rStar), aimed at enhancing reasoning accuracy and factual integrity across large language models (LLMs) for complex, knowledge-intensive tasks such as commonsense and medical reasoning. RARE incorporates two innovative actions within the Monte Carlo Tree Search (MCTS) framework: A6, which generates search queries based on the initial problem statement, performs information retrieval using those queries, and augments reasoning with the retrieved data to formulate the final answer; and A7, which leverages information retrieval specifically for generated sub-questions and re-answers these sub-questions with the relevant contextual information. Additionally, a Retrieval-Augmented Factuality Scorer is proposed to replace the original discriminator, prioritizing reasoning paths that meet high standards of factuality. Experimental results with LLaMA 3.1 show that RARE enables open-source LLMs to achieve competitive performance with top open-source models like GPT-4 and GPT-4o. This research establishes RARE as a scalable solution for improving LLMs in domains where logical coherence and factual integrity are critical.