Robust AI Evaluation through Maximal Lotteries

TL;DR

This paper introduces robust maximal lotteries for evaluating language models, addressing the limitations of traditional ranking methods by accounting for preference heterogeneity and ensuring more reliable, fair comparisons across diverse user groups.

Contribution

It proposes robust lotteries that improve the stability and fairness of model evaluation by optimizing for worst-case preferences, advancing social choice methods in AI evaluation.

Findings

Robust lotteries outperform traditional rankings in stability across diverse preferences.

They provide more reliable win rate guarantees for models.

The approach supports an ecosystem of AI systems serving varied human preferences.

Abstract

The standard way to evaluate language models on subjective tasks is through pairwise comparisons: an annotator chooses the "better" of two responses to a prompt. Leaderboards aggregate these comparisons into a single Bradley-Terry (BT) ranking, forcing heterogeneous preferences into a total order and violating basic social-choice desiderata. In contrast, social choice theory provides an alternative approach called maximal lotteries, which aggregates pairwise preferences without imposing any assumptions on their structure. However, we show that maximal lotteries are highly sensitive to preference heterogeneity and can favor models that severely underperform on specific tasks or user subpopulations. We introduce robust lotteries that optimize worst-case performance under plausible shifts in the preference data. On large-scale preference datasets, robust lotteries provide more reliable win rate guarantees across the annotator distribution and recover a stable set of top-performing models. By moving from rankings to pluralistic sets of winners, robust lotteries offer a principled step toward an ecosystem of complementary AI systems that serve the full spectrum of human preferences.