LAQuer: Localized Attribution Queries in Content-grounded Generation

TL;DR

This paper introduces LAQuer, a task for localized attribution in content-grounded text generation, enabling precise, user-directed source attribution for specific output spans to improve fact-checking and content verification.

Contribution

It proposes the LAQuer task, develops a modeling framework with multiple baselines, and introduces a new evaluation setting to advance localized attribution research.

Findings

LAQuer methods significantly reduce attributed text length.

The framework effectively localizes source spans in summarization and QA tasks.

Prompting LLMs and internal representations both show promise for localization.

Abstract

Grounded text generation models often produce content that deviates from their source material, requiring user verification to ensure accuracy. Existing attribution methods associate entire sentences with source documents, which can be overwhelming for users seeking to fact-check specific claims. In contrast, existing sub-sentence attribution methods may be more precise but fail to align with users' interests. In light of these limitations, we introduce Localized Attribution Queries (LAQuer), a new task that localizes selected spans of generated output to their corresponding source spans, allowing fine-grained and user-directed attribution. We compare two approaches for the LAQuer task, including prompting large language models (LLMs) and leveraging LLM internal representations. We then explore a modeling framework that extends existing attributed text generation methods to LAQuer. We evaluate this framework across two grounded text generation tasks: Multi-document Summarization (MDS) and Long-form Question Answering (LFQA). Our findings show that LAQuer methods significantly reduce the length of the attributed text. Our contributions include: (1) proposing the LAQuer task to enhance attribution usability, (2) suggesting a modeling framework and benchmarking multiple baselines, and (3) proposing a new evaluation setting to promote future research on localized attribution in content-grounded generation.