Preserving Fairness in AI under Domain Shift

TL;DR

This paper introduces an unsupervised domain adaptation algorithm that maintains fairness in AI models despite distributional shifts, reducing the need for costly retraining with annotated data.

Contribution

It presents a novel method for adapting fair models to domain shifts using only unannotated data, addressing a key challenge in fair AI deployment.

Findings

The algorithm effectively preserves fairness under distributional shifts.

Empirical results show improved fairness metrics on multiple datasets.

The approach reduces reliance on costly annotated retraining.

Abstract

Existing algorithms for ensuring fairness in AI use a single-shot training strategy, where an AI model is trained on an annotated training dataset with sensitive attributes and then fielded for utilization. This training strategy is effective in problems with stationary distributions, where both training and testing data are drawn from the same distribution. However, it is vulnerable with respect to distributional shifts in the input space that may occur after the initial training phase. As a result, the time-dependent nature of data can introduce biases into the model predictions. Model retraining from scratch using a new annotated dataset is a naive solution that is expensive and time-consuming. We develop an algorithm to adapt a fair model to remain fair under domain shift using solely new unannotated data points. We recast this learning setting as an unsupervised domain adaptation problem. Our algorithm is based on updating the model such that the internal representation of data remains unbiased despite distributional shifts in the input space. We provide extensive empirical validation on three widely employed fairness datasets to demonstrate the effectiveness of our algorithm.