The Variational Fair Autoencoder

TL;DR

This paper introduces a variational autoencoder that learns invariant representations by disentangling sensitive factors from other data features, using priors and MMD penalties to improve fairness and robustness.

Contribution

It proposes a novel variational autoencoder architecture with priors and MMD penalties to effectively remove sensitive information from learned representations.

Findings

Outperforms previous methods in removing unwanted variation

Produces more invariant and fair latent representations

Efficient training on real datasets

Abstract

We investigate the problem of learning representations that are invariant to certain nuisance or sensitive factors of variation in the data while retaining as much of the remaining information as possible. Our model is based on a variational autoencoding architecture with priors that encourage independence between sensitive and latent factors of variation. Any subsequent processing, such as classification, can then be performed on this purged latent representation. To remove any remaining dependencies we incorporate an additional penalty term based on the "Maximum Mean Discrepancy" (MMD) measure. We discuss how these architectures can be efficiently trained on data and show in experiments that this method is more effective than previous work in removing unwanted sources of variation while maintaining informative latent representations.