Consistency of weighted majority votes

TL;DR

This paper analyzes the consistency of weighted majority voting rules, providing theoretical error bounds and empirical insights, especially when expert competence levels are unknown and must be estimated.

Contribution

It offers a comprehensive statistical learning analysis of weighted expert voting, including new error bounds and methods for unknown competence levels.

Findings

Sharp error estimates for known expert competence levels

Nearly optimal bounds for unknown competence levels

Experimental validation of theoretical results

Abstract

We revisit the classical decision-theoretic problem of weighted expert voting from a statistical learning perspective. In particular, we examine the consistency (both asymptotic and finitary) of the optimal Nitzan-Paroush weighted majority and related rules. In the case of known expert competence levels, we give sharp error estimates for the optimal rule. When the competence levels are unknown, they must be empirically estimated. We provide frequentist and Bayesian analyses for this situation. Some of our proof techniques are non-standard and may be of independent interest. The bounds we derive are nearly optimal, and several challenging open problems are posed. Experimental results are provided to illustrate the theory.