Beyond Arrow: From Impossibility to Possibilities in Multi-Criteria Benchmarking

TL;DR

This paper models multi-criteria benchmarking as a social choice problem, identifying conditions under which aggregation of multiple metrics into a stable, meaningful ranking is feasible, moving beyond traditional impossibility results.

Contribution

It formalizes multi-criteria benchmarking as a social choice problem and identifies conditions that enable stable aggregation of metrics, overcoming prior impossibility limitations.

Findings

Certain preference structures allow stable aggregation of metrics.

Empirical analysis shows real benchmarks often meet these structural conditions.

The approach enables meaningful multi-criteria model rankings.

Abstract

Modern benchmarks such as HELM MMLU account for multiple metrics like accuracy, robustness and efficiency. When trying to turn these metrics into a single ranking, natural aggregation procedures can become incoherent or unstable to changes in the model set. We formalize this aggregation as a social choice problem where each metric induces a preference ranking over models on each dataset, and a benchmark operator aggregates these votes across metrics. While prior work has focused on Arrow's impossibility result, we argue that the impossibility often originates from pathological examples and identify sufficient conditions under which these disappear, and meaningful multi-criteria benchmarking becomes possible. In particular, we deal with three restrictions on the combinations of rankings and prove that on single-peaked, group-separable and distance-restricted preferences, the benchmark operator allows for the construction of well-behaved rankings of the involved models. Empirically, we investigate several modern benchmark suites like HELM MMLU and verify which structural conditions are fulfilled on which benchmark problems.