Online Conformal Abstention for Factuality Control Under Adversarial Bandit Feedback

TL;DR

This paper introduces ExAUL, an online learning framework that improves the reliability of generative models by controlling false responses through conformal abstention, even with partial feedback and adversarial conditions.

Contribution

ExAUL provides a novel online learning method with theoretical guarantees for FDR control using partial feedback, addressing challenges in real-world, adversarial environments.

Findings

ExAUL achieves an $O( oot T)$ regret bound, ensuring effective FDR control.

Empirical results show ExAUL maintains high coverage while controlling false responses.

The framework is effective for large language models in non-stationary, adversarial settings.

Abstract

As interactive generative systems are increasingly deployed in real-world applications, their tendency to generate unreliable or false responses raises serious concerns. Conformal abstention mitigates this risk by ensuring that the system answers only when confident. However, real-world deployments typically provide only partial user feedback (e.g., thumbs up/down) on the selected response and often operate in non-stationary or adversarial environments, for which effective learning methods are largely missing. To bridge this gap, we propose ExAUL, a novel online learning framework for conformal abstention with adversarial and partial feedback. Technically, we introduce (i) a novel conversion lemma}that translates the regret of any bandit algorithm into an FDR bound, and (ii) feedback unlocking, a strategy that exploits the structure of conformal abstention to extract additional learning signals from partial feedback. We prove that ExAUL achieves a regret bound of $O(\sqrt{T \ln |{H}|})$, which translates into an ${O}(\sqrt{T})$ bound on FDR risk control, matching the controllability of full-information settings despite receiving only partial feedback. While applicable to general generative tasks, we demonstrate the efficacy of ExAUL for ensuring the reliability of Large Language Models (LLMs) through empirical validation on question-answering tasks across diverse non-stationary and adversarial settings. Our results demonstrate that ExAUL robustly controls the FDR while maintaining competitive answering coverage.