Conditional Factuality Controlled LLMs with Generalization Certificates via Conformal Sampling

TL;DR

This paper introduces Conditional Factuality Control (CFC), a conformal prediction framework for LLMs that provides set-valued outputs with conditional coverage guarantees, improving reliability and efficiency over existing methods.

Contribution

The paper proposes CFC, a novel post-hoc conformal method that achieves conditional coverage guarantees and better sample efficiency compared to traditional marginal conformal prediction.

Findings

CFC attains near-target coverage across various benchmarks.

CFC produces smaller prediction sets than baseline methods.

CFC-PAC provides finite-sample certificates for coverage deviation.

Abstract

Large language models (LLMs) need reliable test-time control of hallucinations. Existing conformal methods for LLMs typically provide only \emph{marginal} guarantees and rely on a single global threshold, which can under-cover hard prompts, over-cover easy ones, and produce oversized prediction sets. We propose \emph{Conditional Factuality Control} (CFC), a post-hoc conformal framework that returns \emph{set-valued} outputs with \emph{conditional} coverage guarantees. CFC defines a continuous, feature-conditional acceptance threshold through augmented quantile regression on a latent ``success'' score, and deploys it through a fixed-point threshold rule at inference time. Theoretically, we show that CFC satisfies a conditional coverage guarantee under exchangeability and analyze its \emph{efficiency}, proving that, under mild assumptions on the score distributions, the conditional rule is strictly more sample-efficient than marginal conformal prediction at the same target coverage. We further derive a PAC-style variant, CFC-PAC, which shrinks the nominal risk level based on a stability bound, yielding a finite-sample certificate that the conditional miscoverage deviates from the target by at most $O(\sqrt{\log(1/\delta)/N})$. Empirically, on synthetic data, real-world reasoning and QA benchmarks, and a Flickr8k VLM setting, CFC and CFC-PAC consistently attain near-target coverage across difficulty groups while using smaller prediction sets than CP and non-CP baselines.