Stability of Weighted Majority Voting under Estimated Weights

TL;DR

This paper analyzes the stability of weighted majority voting when trustworthiness is estimated rather than known, proving that correctness stability holds but optimality stability does not, and exploring sensitivity to trust estimates.

Contribution

It introduces formal properties of unbiased trust estimates in WMV, proving correctness stability and analyzing the instability of optimality.

Findings

Correctness stability always holds for unbiased trust estimates.

Optimality stability generally does not hold, leading to potential decision suboptimality.

The paper provides bounds and sensitivity analysis for trust estimation errors.

Abstract

Weighted Majority Voting (WMV) is a well-known optimal decision rule for collective decision making, given the probability of sources to provide accurate information (trustworthiness). However, in reality, the trustworthiness is not a known quantity to the decision maker - they have to rely on an estimate called trust. A (machine learning) algorithm that computes trust is called unbiased when it has the property that it does not systematically overestimate or underestimate the trustworthiness. To formally analyse the uncertainty to the decision process, we introduce and analyse two important properties of such unbiased trust values: stability of correctness and stability of optimality. Stability of correctness means that the decision accuracy that the decision maker believes they achieved is equal to the actual accuracy. We prove stability of correctness holds. Stability of optimality means that the decisions made based on trust, are equally good as they would have been if they were based on trustworthiness. Stability of optimality does not hold. We analyse the difference between the two, and bounds thereon. We also present an overview of how sensitive decision correctness is to changes in trust and trustworthiness.