Think Through Uncertainty: Improving Long-Form Generation Factuality via Reasoning Calibration

TL;DR

This paper introduces CURE, a framework that enhances long-form generation factuality by enabling LLMs to reason about uncertainty at the claim level, leading to more accurate and calibrated outputs.

Contribution

CURE is the first approach to teach LLMs claim-level reasoning about uncertainty, improving factual accuracy and calibration in long-form generation.

Findings

CURE improves claim-level accuracy by up to 39.9% on Biography generation.

CURE increases AUROC by 16.0% on FactBench, indicating better calibration.

CURE maintains factual recall while enhancing factuality.

Abstract

Large language models (LLMs) often hallucinate in long-form generation. Existing approaches mainly improve factuality through post-hoc revision or reinforcement learning (RL) with correctness-based rewards, but they do not teach the model to estimate which parts of its generation are reliable. As a result, models may still state incorrect claims confidently in their responses. Recent advances in reasoning have significantly improved LLM performance, and have been leveraged to estimate confidence by incorporating calibration into RL objectives. However, existing approaches remain limited to a single scalar confidence for the entire response, which is insufficient for long-form generation where uncertainty varies across individual claims. To mitigate this problem, we propose CURE, a framework that improves long-form factuality by teaching LLMs to reason about uncertainty at the claim level. We first introduce a Claim-Aware Reasoning Protocol, which structures outputs into atomic claims paired with explicit confidence estimates. We then develop a multi-stage training pipeline that aligns model confidence with claims' correctness and then optimizes on factuality. The resulting calibrated confidence further enables selective prediction, allowing the model to abstain from uncertain claims at inference time. Experiments on four long-form factuality benchmarks show that CURE consistently improves factual accuracy over competitive supervised and RL baselines, while maintaining factual recall. In particular, it improves claim-level accuracy by up to 39.9% on Biography generation. These gains are accompanied by improved calibration, as reflected by a 16.0% increase in AUROC on FactBench.