Learning Unbiased Representations via R\'enyi Minimization

TL;DR

This paper introduces an adversarial method using the Hirschfeld-Gebelein-Renyi (HGR) maximal correlation coefficient to learn unbiased representations that effectively mitigate bias related to sensitive attributes in machine learning models.

Contribution

It proposes a novel adversarial algorithm leveraging HGR maximal correlation for bias mitigation, capturing complex non-linear dependencies more effectively than previous measures.

Findings

Significant bias reduction demonstrated empirically

Outperforms existing fairness algorithms

Effective in capturing non-linear dependencies

Abstract

In recent years, significant work has been done to include fairness constraints in the training objective of machine learning algorithms. Many state-of the-art algorithms tackle this challenge by learning a fair representation which captures all the relevant information to predict the output Y while not containing any information about a sensitive attribute S. In this paper, we propose an adversarial algorithm to learn unbiased representations via the Hirschfeld-Gebelein-Renyi (HGR) maximal correlation coefficient. We leverage recent work which has been done to estimate this coefficient by learning deep neural network transformations and use it as a minmax game to penalize the intrinsic bias in a multi dimensional latent representation. Compared to other dependence measures, the HGR coefficient captures more information about the non-linear dependencies with the sensitive variable, making the algorithm more efficient in mitigating bias in the representation. We empirically evaluate and compare our approach and demonstrate significant improvements over existing works in the field.