Understanding Fairness-Accuracy Trade-offs in Machine Learning Models: Does Promoting Fairness Undermine Performance?

TL;DR

This paper investigates the fairness-accuracy trade-off in ML models for university admissions, demonstrating that ML models can outperform humans in fairness consistency while maintaining high accuracy, supporting hybrid decision approaches.

Contribution

It introduces a new fairness consistency metric and compares ML models with human evaluators, showing ML's superior fairness performance in real-world admissions data.

Findings

ML models outperform humans in fairness consistency by 14-19%.

ML models maintain high accuracy in admissions decisions.

Hybrid approaches can leverage ML fairness and human judgment.

Abstract

Fairness in both Machine Learning (ML) predictions and human decision-making is essential, yet both are susceptible to different forms of bias, such as algorithmic and data-driven in ML, and cognitive or subjective in humans. In this study, we examine fairness using a real-world university admissions dataset comprising 870 applicant profiles, leveraging three ML models: XGB, Bi-LSTM, and KNN, alongside BERT embeddings for textual features. To evaluate individual fairness, we introduce a consistency metric that quantifies agreement in decisions among ML models and human experts with diverse backgrounds. Our analysis reveals that ML models surpass human evaluators in fairness consistency by margins ranging from 14.08\% to 18.79\%. Our findings highlight the potential of using ML to enhance fairness in admissions while maintaining high accuracy, advocating a hybrid approach combining human judgement and ML models.