Favi-Score: A Measure for Favoritism in Automated Preference Ratings for Generative AI Evaluation

TL;DR

This paper introduces Favi-Score, a new metric to quantify favoritism in automated preference ratings for generative AI, highlighting issues with current metrics that skew evaluations and proposing a more comprehensive assessment approach.

Contribution

The paper formally defines favoritism in preference metrics and introduces Favi-Score to measure it, addressing a gap in current evaluation methods for generative AI.

Findings

Current metrics often favor specific systems, skewing evaluations.

Favoritism in metrics correlates with errors in system ranking.

Evaluating both sign accuracy and favoritism provides a more balanced assessment.

Abstract

Generative AI systems have become ubiquitous for all kinds of modalities, which makes the issue of the evaluation of such models more pressing. One popular approach is preference ratings, where the generated outputs of different systems are shown to evaluators who choose their preferences. In recent years the field shifted towards the development of automated (trained) metrics to assess generated outputs, which can be used to create preference ratings automatically. In this work, we investigate the evaluation of the metrics themselves, which currently rely on measuring the correlation to human judgments or computing sign accuracy scores. These measures only assess how well the metric agrees with the human ratings. However, our research shows that this does not tell the whole story. Most metrics exhibit a disagreement with human system assessments which is often skewed in favor of particular text generation systems, exposing a degree of favoritism in automated metrics. This paper introduces a formal definition of favoritism in preference metrics, and derives the Favi-Score, which measures this phenomenon. In particular we show that favoritism is strongly related to errors in final system rankings. Thus, we propose that preference-based metrics ought to be evaluated on both sign accuracy scores and favoritism.