Detecting Systematic Weaknesses in Vision Models along Predefined Human-Understandable Dimensions

TL;DR

This paper introduces a new method combining foundation models and combinatorial search to identify human-understandable weaknesses in vision models, validated on synthetic and real datasets.

Contribution

The paper presents a novel algorithm that leverages foundation models for zero-shot classification to find systematic weaknesses aligned with human-understandable dimensions in image data.

Findings

Successfully identifies weaknesses in state-of-the-art vision models

Effective on both synthetic and real-world datasets

Addresses noise in semantic metadata

Abstract

Slice discovery methods (SDMs) are prominent algorithms for finding systematic weaknesses in DNNs. They identify top-k semantically coherent slices/subsets of data where a DNN-under-test has low performance. For being directly useful, slices should be aligned with human-understandable and relevant dimensions, which, for example, are defined by safety and domain experts as part of the operational design domain (ODD). While SDMs can be applied effectively on structured data, their application on image data is complicated by the lack of semantic metadata. To address these issues, we present an algorithm that combines foundation models for zero-shot image classification to generate semantic metadata with methods for combinatorial search to find systematic weaknesses in images. In contrast to existing approaches, ours identifies weak slices that are in line with pre-defined human-understandable dimensions. As the algorithm includes foundation models, its intermediate and final results may not always be exact. Therefore, we include an approach to address the impact of noisy metadata. We validate our algorithm on both synthetic and real-world datasets, demonstrating its ability to recover human-understandable systematic weaknesses. Furthermore, using our approach, we identify systematic weaknesses of multiple pre-trained and publicly available state-of-the-art computer vision DNNs.