Epistemic Filtering and Collective Hallucination: A Jury Theorem for Confidence-Calibrated Agents

TL;DR

This paper introduces a probabilistic framework where agents learn their reliability and abstain when unsure, enhancing collective decision accuracy beyond classical voting theorems, with applications to AI safety.

Contribution

It generalizes the Condorcet Jury Theorem to a sequential, confidence-based setting allowing agents to abstain, supported by theoretical bounds and empirical validation.

Findings

Derived a lower bound on group success probability with selective participation.

Proved that confidence gating extends asymptotic guarantees of classical voting theorems.

Validated bounds through Monte Carlo simulations.

Abstract

We investigate the collective accuracy of heterogeneous agents who learn to estimate their own reliability over time and selectively abstain from voting. While classical epistemic voting results, such as the \textit{Condorcet Jury Theorem} (CJT), assume fixed participation, real-world aggregation often benefits from allowing agents to say ``I don't know.'' We propose a probabilistic framework where agents engage in a \textit{calibration} phase, updating beliefs about their own fixed competence, before facing a final confidence gate that determines whether to vote or abstain. We derive a non-asymptotic lower bound on the group's success probability and prove that this \textit{selective participation} generalizes the asymptotic guarantees of the CJT to a sequential, confidence-gated setting. Empirically, we validate these bounds via Monte Carlo simulations. While our results are general, we discuss their potential application to AI safety, outlining how this framework can mitigate \textit{hallucinations} in collective LLM decision-making.