Identification of Systematic Errors of Image Classifiers on Rare Subgroups

TL;DR

This paper introduces PromptAttack, a method leveraging text-to-image models and combinatorial testing to identify rare subgroups where image classifiers perform poorly, revealing systematic errors without subgroup annotations.

Contribution

The paper proposes PromptAttack, a novel approach combining prompt-based search and combinatorial testing to detect systematic errors in image classifiers on rare, unannotated subgroups.

Findings

PromptAttack accurately identifies systematic errors in controlled settings.

It uncovers novel errors on rare subgroups in ImageNet classifiers.

The method enhances understanding of classifier robustness and fairness.

Abstract

Despite excellent average-case performance of many image classifiers, their performance can substantially deteriorate on semantically coherent subgroups of the data that were under-represented in the training data. These systematic errors can impact both fairness for demographic minority groups as well as robustness and safety under domain shift. A major challenge is to identify such subgroups with subpar performance when the subgroups are not annotated and their occurrence is very rare. We leverage recent advances in text-to-image models and search in the space of textual descriptions of subgroups ("prompts") for subgroups where the target model has low performance on the prompt-conditioned synthesized data. To tackle the exponentially growing number of subgroups, we employ combinatorial testing. We denote this procedure as PromptAttack as it can be interpreted as an adversarial attack in a prompt space. We study subgroup coverage and identifiability with PromptAttack in a controlled setting and find that it identifies systematic errors with high accuracy. Thereupon, we apply PromptAttack to ImageNet classifiers and identify novel systematic errors on rare subgroups.