Is it Still Fair? A Comparative Evaluation of Fairness Algorithms through the Lens of Covariate Drift

TL;DR

This paper investigates how natural data distribution changes affect the fairness of machine learning models, revealing that drift can significantly impair fairness and that current methods often overlook this issue.

Contribution

The study provides a comprehensive analysis of the impact of data drift on fairness algorithms, highlighting gaps in existing research and offering policy implications.

Findings

Data drift can cause significant fairness deterioration in models.

The size and direction of data drift are not directly linked to unfairness.

Fairness algorithm performance is affected by data drift, which is often ignored.

Abstract

Over the last few decades, machine learning (ML) applications have grown exponentially, yielding several benefits to society. However, these benefits are tempered with concerns of discriminatory behaviours exhibited by ML models. In this regard, fairness in machine learning has emerged as a priority research area. Consequently, several fairness metrics and algorithms have been developed to mitigate against discriminatory behaviours that ML models may possess. Yet still, very little attention has been paid to the problem of naturally occurring changes in data patterns (\textit{aka} data distributional drift), and its impact on fairness algorithms and metrics. In this work, we study this problem comprehensively by analyzing 4 fairness-unaware baseline algorithms and 7 fairness-aware algorithms, carefully curated to cover the breadth of its typology, across 5 datasets including public and proprietary data, and evaluated them using 3 predictive performance and 10 fairness metrics. In doing so, we show that (1) data distributional drift is not a trivial occurrence, and in several cases can lead to serious deterioration of fairness in so-called fair models; (2) contrary to some existing literature, the size and direction of data distributional drift is not correlated to the resulting size and direction of unfairness; and (3) choice of, and training of fairness algorithms is impacted by the effect of data distributional drift which is largely ignored in the literature. Emanating from our findings, we synthesize several policy implications of data distributional drift on fairness algorithms that can be very relevant to stakeholders and practitioners.