Precise Information Control in Long-Form Text Generation

TL;DR

This paper introduces Precise Information Control (PIC), a new task and benchmark for long-form text generation that emphasizes faithfulness to input claims, revealing current models' hallucination issues and proposing a training framework to improve factual accuracy.

Contribution

The paper formulates PIC as a new task for grounded long-form generation, creates PIC-Bench benchmark, and develops a weakly supervised training method to significantly enhance model faithfulness.

Findings

State-of-the-art LMs hallucinate over 70% of the time against input claims.

The PIC-LM improves F1 score from 69.1% to 91.0% in the full PIC setting.

PIC-LM enhances factual accuracy metrics in downstream tasks.

Abstract

A central challenge in language models (LMs) is faithfulness hallucination: the generation of information unsubstantiated by input context. To study this problem, we propose Precise Information Control (PIC), a new task formulation that requires models to generate long-form outputs grounded in a provided set of short self-contained statements, without adding any unsupported ones. PIC includes a full setting that tests a model's ability to include exactly all input claims, and a partial setting that requires the model to selectively incorporate only relevant claims. We present PIC-Bench, a benchmark of eight long-form generation tasks (e.g., summarization, biography generation) adapted to the PIC setting, where LMs are supplied with well-formed, verifiable input claims. Our evaluation of a range of open and proprietary LMs on PIC-Bench reveals that, surprisingly, state-of-the-art LMs still hallucinate against user-provided input in over 70% of generations. To alleviate this lack of faithfulness, we introduce a post-training framework that uses a weakly supervised preference data construction method to train an 8B PIC-LM with stronger PIC ability--improving from 69.1% to 91.0% F1 in the full PIC setting. When integrated into end-to-end factual generation pipelines, PIC-LM improves exact match recall by 17.1% on ambiguous QA with retrieval, and factual precision by 30.5% on a birthplace fact-checking task, underscoring the potential of precisely grounded generation.