From Efficiency to Equity: Measuring Fairness in Preference Learning

TL;DR

This paper presents a new framework for measuring fairness in preference learning models, inspired by economic theories, and demonstrates its effectiveness on datasets highlighting inequalities in AI systems.

Contribution

Introduces a novel fairness evaluation framework for preference learning models based on economic inequality metrics, with validation on real datasets and mitigation techniques.

Findings

Variations in model performance across users highlight potential biases.

Economic-inspired metrics effectively quantify epistemic fairness.

Pre-processing and in-processing methods can reduce inequalities.

Abstract

As AI systems, particularly generative models, increasingly influence decision-making, ensuring that they are able to fairly represent diverse human preferences becomes crucial. This paper introduces a novel framework for evaluating epistemic fairness in preference learning models inspired by economic theories of inequality and Rawlsian justice. We propose metrics adapted from the Gini Coefficient, Atkinson Index, and Kuznets Ratio to quantify fairness in these models. We validate our approach using two datasets: a custom visual preference dataset (AI-EDI-Space) and the Jester Jokes dataset. Our analysis reveals variations in model performance across users, highlighting potential epistemic injustices. We explore pre-processing and in-processing techniques to mitigate these inequalities, demonstrating a complex relationship between model efficiency and fairness. This work contributes to AI ethics by providing a framework for evaluating and improving epistemic fairness in preference learning models, offering insights for developing more inclusive AI systems in contexts where diverse human preferences are crucial.