COIN: Uncertainty-Guarding Selective Question Answering for Foundation Models with Provable Risk Guarantees

TL;DR

COIN introduces a statistically grounded framework for selective question answering in foundation models, ensuring controlled false discovery rates while maximizing answer retention and applicability across various tasks.

Contribution

The paper presents COIN, a novel uncertainty-guarding selection method that calibrates thresholds for FDR control, improving the reliability and efficiency of model-generated answers.

Findings

COIN effectively controls FDR with high probability.

It significantly increases sample retention compared to existing methods.

The approach is robust across different tasks and data limitations.

Abstract

Uncertainty quantification (UQ) for foundation models is essential to identify and mitigate potential hallucinations in automatically generated text. However, heuristic UQ approaches lack formal guarantees for key metrics such as the false discovery rate (FDR) in selective prediction. Previous work adopts the split conformal prediction (SCP) framework to ensure desired coverage of admissible answers by constructing prediction sets, but these sets often contain incorrect candidates, limiting their practical utility. To address this, we propose COIN, an uncertainty-guarding selection framework that calibrates statistically valid thresholds to filter a single generated answer per question under user-specified FDR constraints. COIN estimates the empirical error rate on a calibration set and applies confidence interval methods such as Clopper-Pearson to establish a high-probability upper bound on the true error rate (i.e., FDR). This enables the selection of the largest uncertainty threshold that ensures FDR control on test data while significantly increasing sample retention. We demonstrate COIN's robustness in risk control, strong test-time power in retaining admissible answers, and predictive efficiency under limited calibration data across both general and multimodal text generation tasks. Furthermore, we show that employing alternative upper bound constructions and UQ strategies can further boost COIN's power performance, which underscores its extensibility and adaptability to diverse application scenarios.