Transfer of Machine Learning Fairness across Domains

TL;DR

This paper explores how to transfer fairness in machine learning models across different domains, providing theoretical guarantees and practical methods to improve fairness in data-sparse target domains.

Contribution

It introduces a theoretical framework and modeling approach for transferring fairness across domains, addressing challenges of limited data and domain shifts.

Findings

Theoretical guarantees for fairness transfer across domains.

Empirical validation showing improved fairness metrics.

Effective in data-sparse target domains.

Abstract

If our models are used in new or unexpected cases, do we know if they will make fair predictions? Previously, researchers developed ways to debias a model for a single problem domain. However, this is often not how models are trained and used in practice. For example, labels and demographics (sensitive attributes) are often hard to observe, resulting in auxiliary or synthetic data to be used for training, and proxies of the sensitive attribute to be used for evaluation of fairness. A model trained for one setting may be picked up and used in many others, particularly as is common with pre-training and cloud APIs. Despite the pervasiveness of these complexities, remarkably little work in the fairness literature has theoretically examined these issues. We frame all of these settings as domain adaptation problems: how can we use what we have learned in a source domain to debias in a new target domain, without directly debiasing on the target domain as if it is a completely new problem? We offer new theoretical guarantees of improving fairness across domains, and offer a modeling approach to transfer to data-sparse target domains. We give empirical results validating the theory and showing that these modeling approaches can improve fairness metrics with less data.