When Can Voting Help, Hurt, or Change Course? Exact Structure of Binary Test-Time Aggregation

TL;DR

This paper analyzes the exact structure of voting in ensemble predictions, revealing complex behaviors like nonmonotonicity and multiple trend changes driven by latent correctness distributions.

Contribution

It introduces a novel theoretical framework linking voting curves to signed voting signatures, providing a complete characterization of voting behavior.

Findings

Voting curves can exhibit nonmonotone behavior and multiple trend changes.

The full voting signature is uniquely determined by the voting curve.

Shape phenomena and nonidentifiability are explained through the signature and curve relationship.

Abstract

Majority voting is one of the few black-box interventions that can improve a fixed stochastic predictor: repeated access can be cheaper than changing a high-capability model. Classical fixed-competence theory makes this intervention look monotone -- more votes help above the majority threshold and hurt below it. We show that this picture is fundamentally incomplete. Under the de Finetti representation for exchangeable repeated correctness, voting is governed by a latent distribution of per-example correctness probabilities. Even simple latent mixtures can generate sharply different voting curves, including nonmonotone behavior and, in an explicit construction, infinitely many trend changes. The full latent law determines the curve, but the curve does not determine the law. The exact object recovered by voting is a signed voting signature: at each binomial variance scale, it records excess latent mass above rather than below the majority threshold. Our main theorem proves that the complete odd-budget curve and this signature are equivalent: the curve increments are signed Hausdorff moments, and the full curve recovers the signature uniquely. This viewpoint explains shape phenomena, branch-symmetric nonidentifiability, realizability, variation, and endpoint rates. It also separates estimation regimes: direct per-example success-probability information targets the full signature, whereas fixed-depth grouped labels reveal only a finite prefix.