The Spotlight: A General Method for Discovering Systematic Errors in Deep Learning Models

TL;DR

This paper presents a general method called 'The Spotlight' to discover systematic errors in deep learning models by analyzing hidden representations, revealing semantically meaningful weaknesses across diverse applications.

Contribution

Introduces a novel technique leveraging hidden layer representations to identify systematic errors not tied to explicit labels in deep learning models.

Findings

Effective in diverse domains like vision, NLP, and recommender systems.

Identifies semantically meaningful error regions.

Uncovers model weaknesses beyond explicit label groups.

Abstract

Supervised learning models often make systematic errors on rare subsets of the data. When these subsets correspond to explicit labels in the data (e.g., gender, race) such poor performance can be identified straightforwardly. This paper introduces a method for discovering systematic errors that do not correspond to such explicitly labelled subgroups. The key idea is that similar inputs tend to have similar representations in the final hidden layer of a neural network. We leverage this structure by "shining a spotlight" on this representation space to find contiguous regions where the model performs poorly. We show that the spotlight surfaces semantically meaningful areas of weakness in a wide variety of existing models spanning computer vision, NLP, and recommender systems.