UDIS: Unsupervised Discovery of Bias in Deep Visual Recognition Models

TL;DR

UDIS is an unsupervised method that detects and analyzes failure modes in deep visual models by clustering dataset embeddings and visualizing low-performing subpopulations, reducing reliance on costly annotations.

Contribution

Introduces UDIS, a novel unsupervised algorithm for discovering and analyzing bias and failure modes in deep visual recognition models without requiring protected attribute annotations.

Findings

Successfully identifies failure modes in CelebA and MSCOCO datasets.

Effectively visualizes systematic failures using class-activation maps.

Reduces need for expensive annotation in bias detection.

Abstract

Deep learning models have been shown to learn spurious correlations from data that sometimes lead to systematic failures for certain subpopulations. Prior work has typically diagnosed this by crowdsourcing annotations for various protected attributes and measuring performance, which is both expensive to acquire and difficult to scale. In this work, we propose UDIS, an unsupervised algorithm for surfacing and analyzing such failure modes. UDIS identifies subpopulations via hierarchical clustering of dataset embeddings and surfaces systematic failure modes by visualizing low performing clusters along with their gradient-weighted class-activation maps. We show the effectiveness of UDIS in identifying failure modes in models trained for image classification on the CelebA and MSCOCO datasets.