On Adversarial Bias and the Robustness of Fair Machine Learning

TL;DR

This paper investigates how adversarial attacks can compromise the robustness of fair machine learning models, especially those using equalized odds, revealing vulnerabilities that can reduce accuracy and fairness despite fairness constraints.

Contribution

It provides an analysis of data poisoning attacks against group-based fair ML, highlighting conflicts between fairness and robustness, with empirical evaluation across multiple algorithms and datasets.

Findings

Adversarial sampling and labeling attacks can significantly reduce test accuracy.

Such attacks can increase the fairness gap on test data.

Fair models can still be vulnerable despite satisfying fairness constraints on training data.

Abstract

Optimizing prediction accuracy can come at the expense of fairness. Towards minimizing discrimination against a group, fair machine learning algorithms strive to equalize the behavior of a model across different groups, by imposing a fairness constraint on models. However, we show that giving the same importance to groups of different sizes and distributions, to counteract the effect of bias in training data, can be in conflict with robustness. We analyze data poisoning attacks against group-based fair machine learning, with the focus on equalized odds. An adversary who can control sampling or labeling for a fraction of training data, can reduce the test accuracy significantly beyond what he can achieve on unconstrained models. Adversarial sampling and adversarial labeling attacks can also worsen the model's fairness gap on test data, even though the model satisfies the fairness constraint on training data. We analyze the robustness of fair machine learning through an empirical evaluation of attacks on multiple algorithms and benchmark datasets.