Truth-tracking via Approval Voting: Size Matters

TL;DR

This paper introduces a new approval voting model that accounts for vote reliability based on ballot size, improving ground truth detection in noisy settings, with empirical validation on image annotation datasets.

Contribution

It proposes approval voting variants of the Mallows model that weight votes by reliability, a novel approach for epistemic social choice with noisy signals.

Findings

Rules based on the new noise model outperform standard approval voting

The Condorcet noise model variant achieves the best performance

Empirical results on image datasets validate the approach

Abstract

Epistemic social choice aims at unveiling a hidden ground truth given votes, which are interpreted as noisy signals about it. We consider here a simple setting where votes consist of approval ballots: each voter approves a set of alternatives which they believe can possibly be the ground truth. Based on the intuitive idea that more reliable votes contain fewer alternatives, we define several noise models that are approval voting variants of the Mallows model. The likelihood-maximizing alternative is then characterized as the winner of a weighted approval rule, where the weight of a ballot decreases with its cardinality. We have conducted an experiment on three image annotation datasets; they conclude that rules based on our noise model outperform standard approval voting; the best performance is obtained by a variant of the Condorcet noise model.