Think&Cite: Improving Attributed Text Generation with Self-Guided Tree Search and Progress Reward Modeling

TL;DR

This paper introduces Think&Cite, a novel attributed text generation framework that employs self-guided tree search and progress reward modeling to enhance factual accuracy and reasoning in large language models.

Contribution

It presents a new multi-step reasoning approach with Self-Guided Monte Carlo Tree Search and progress reward modeling to improve attributed text generation.

Findings

Significant performance improvement over baselines on three datasets.

Effective use of self-reflection for guiding tree search.

Enhanced factual consistency and reasoning in generated texts.

Abstract

Despite their outstanding capabilities, large language models (LLMs) are prone to hallucination and producing factually incorrect information. This challenge has spurred efforts in attributed text generation, which prompts LLMs to generate content with supporting evidence. In this paper, we propose a novel framework, called Think&Cite, and formulate attributed text generation as a multi-step reasoning problem integrated with search. Specifically, we propose Self-Guided Monte Carlo Tree Search (SG-MCTS), which capitalizes on the self-reflection capability of LLMs to reason about the intermediate states of MCTS for guiding the tree expansion process. To provide reliable and comprehensive feedback, we introduce Progress Reward Modeling to measure the progress of tree search from the root to the current state from two aspects, i.e., generation and attribution progress. We conduct extensive experiments on three datasets and the results show that our approach significantly outperforms baseline approaches.