When Should LLMs Be Less Specific? Selective Abstraction for Reliable Long-Form Text Generation

TL;DR

This paper introduces Selective Abstraction, a method for LLMs to improve reliability in long-form generation by selectively reducing detail in uncertain content, thereby balancing factual accuracy and informativeness.

Contribution

The paper proposes Atom-wise Selective Abstraction, formalizes the framework, and demonstrates its effectiveness across multiple models and benchmarks, outperforming existing methods.

Findings

Atom-wise SA improves factual correctness and coverage.

Reduces specificity to enhance reliability.

Outperforms claim removal baseline by up to 27.73% in AURC.

Abstract

LLMs are widely used, yet they remain prone to factual errors that erode user trust and limit adoption in high-risk settings. One approach to mitigate this risk is to equip models with uncertainty estimation mechanisms that abstain when confidence is low. However, this binary "all-or-nothing" approach is excessively restrictive in long-form settings, often discarding valuable information. We introduce Selective Abstraction (SA), a framework that enables LLMs to trade specificity for reliability by selectively reducing the detail of uncertain content. We first formalize SA through the lenses of selective risk and coverage. We then propose Atom-wise Selective Abstraction, a claim-level instantiation that decomposes responses into atomic claims (short, self-contained statements each expressing a single fact) and replaces uncertain atoms with higher confidence, less specific abstractions. To evaluate this framework, we develop a novel end-to-end pipeline for open-ended generation that instantiates risk as factual correctness and measures coverage using an information-theoretic measure of retained information. Across six open-source models on the FactScore and LongFact-Objects benchmarks, atom-wise SA consistently outperforms existing baselines, improving the area under the risk-coverage curve (AURC) by up to 27.73% over claim removal, demonstrating that reducing specificity can boost accuracy and reliability while preserving most of their original meaning.