Shortcut Detection with Variational Autoencoders

TL;DR

This paper introduces a novel method using variational autoencoders to detect spurious correlations, or shortcuts, in image and audio datasets, aiding in the development of more robust machine learning models.

Contribution

The work presents a new approach leveraging VAE feature disentanglement to semi-automatically identify shortcuts in datasets, addressing a scarcely explored problem.

Findings

Successfully identified previously unknown shortcuts in datasets

Demonstrated applicability on multiple real-world datasets

Showed effectiveness in revealing spurious correlations

Abstract

For real-world applications of machine learning (ML), it is essential that models make predictions based on well-generalizing features rather than spurious correlations in the data. The identification of such spurious correlations, also known as shortcuts, is a challenging problem and has so far been scarcely addressed. In this work, we present a novel approach to detect shortcuts in image and audio datasets by leveraging variational autoencoders (VAEs). The disentanglement of features in the latent space of VAEs allows us to discover feature-target correlations in datasets and semi-automatically evaluate them for ML shortcuts. We demonstrate the applicability of our method on several real-world datasets and identify shortcuts that have not been discovered before.