Ground Every Sentence: Improving Retrieval-Augmented LLMs with Interleaved Reference-Claim Generation

TL;DR

This paper introduces ReClaim, a fine-grained attribution method for retrieval-augmented LLMs that generates sentence-level citations step-by-step, significantly improving citation accuracy and verifiability in knowledge-intensive tasks.

Contribution

ReClaim is the first method to produce sentence-level citations through interleaved reference and claim generation, enhancing the verifiability of RAG systems.

Findings

Achieves 90% citation accuracy in experiments

Outperforms coarse-grained attribution methods

Enhances credibility and verifiability of LLM responses

Abstract

Retrieval-Augmented Generation (RAG) has been widely adopted to enhance Large Language Models (LLMs) in knowledge-intensive tasks. To enhance credibility and verifiability in RAG systems, Attributed Text Generation (ATG) is proposed, which provides citations to retrieval knowledge in LLM-generated responses. Prior methods mainly adopt coarse-grained attributions, with passage-level or paragraph-level references or citations, which fall short in verifiability. This paper proposes ReClaim (Refer & Claim), a fine-grained ATG method that alternates the generation of references and answers step by step. Different from previous coarse-grained attribution, ReClaim provides sentence-level citations in long-form question-answering tasks. With extensive experiments, we verify the effectiveness of ReClaim in extensive settings, achieving a citation accuracy rate of 90%.