Fair Inputs and Fair Outputs: The Incompatibility of Fairness in Privacy and Accuracy

TL;DR

This paper investigates the inherent conflicts between fairness in input data collection, privacy, and output accuracy in algorithmic decision systems, revealing that these fairness properties are often incompatible in practice.

Contribution

It introduces formal properties for input fairness, analyzes their interaction with output fairness, and provides an algorithm to detect trade-offs in real datasets.

Findings

Fairness in inputs and outputs are often incompatible.

The trade-off between privacy, fairness, and accuracy is common in real data.

An algorithm can verify the existence of these trade-offs in datasets.

Abstract

Fairness concerns about algorithmic decision-making systems have been mainly focused on the outputs (e.g., the accuracy of a classifier across individuals or groups). However, one may additionally be concerned with fairness in the inputs. In this paper, we propose and formulate two properties regarding the inputs of (features used by) a classifier. In particular, we claim that fair privacy (whether individuals are all asked to reveal the same information) and need-to-know (whether users are only asked for the minimal information required for the task at hand) are desirable properties of a decision system. We explore the interaction between these properties and fairness in the outputs (fair prediction accuracy). We show that for an optimal classifier these three properties are in general incompatible, and we explain what common properties of data make them incompatible. Finally we provide an algorithm to verify if the trade-off between the three properties exists in a given dataset, and use the algorithm to show that this trade-off is common in real data.