Theoretical Foundations of {\delta}-margin Majority Voting

TL;DR

This paper develops a rigorous theoretical framework for { extdelta}-margin majority voting, enabling precise design and analysis of consensus-based decision systems in high-stakes machine learning applications.

Contribution

It introduces a comprehensive theoretical model using Markov chains and Gambler's Ruin theory, providing closed-form formulas and Bayesian extensions for designing { extdelta}-margin voting systems.

Findings

Close agreement between theoretical predictions and real-world data

Framework enables model-based design instead of trial-and-error

Supports real-time monitoring and cost calibration of voting processes

Abstract

In high-stakes ML applications such as fraud detection, medical diagnostics, and content moderation, practitioners rely on consensus-based approaches to control prediction quality. A particularly valuable technique -- {\delta}\delta {\delta}-margin majority voting -- collects votes sequentially until one label exceeds alternatives by a threshold {\delta}\delta {\delta}, offering stronger confidence than simple majority voting. Despite widespread adoption, this approach has lacked rigorous theoretical foundations, leaving practitioners reliant on heuristics for key metrics like expected accuracy and cost. This paper establishes a comprehensive theoretical framework for {\delta}\delta {\delta}-margin majority voting by formulating it as an absorbing Markov chain and leveraging Gambler's Ruin theory. Our contributions form a practical \emph{design calculus} for {\delta}\delta {\delta}-margin voting: (1)~Closed-form expressions for consensus accuracy, expected voting duration, variance, and the stopping-time PMF, enabling model-based design rather than trial-and-error. (2)~A Bayesian extension handling uncertainty in worker accuracy, supporting real-time monitoring of expected quality and cost as votes arrive, with single-Beta and mixture-of-Betas priors. (3)~Cost-calibration methods for achieving equivalent quality across worker pools with different accuracies and for setting payment rates accordingly. We validate our predictions on two real-world datasets, demonstrating close agreement between theory and observed outcomes. The framework gives practitioners a rigorous toolkit for designing {\delta}\delta {\delta}-margin voting processes, replacing ad-hoc experimentation with model-based design where quality control and cost transparency are essential.