Failing Conceptually: Concept-Based Explanations of Dataset Shift

TL;DR

This paper introduces Concept Bottleneck Shift Detection (CBSD), an explainable method that identifies and ranks high-level concepts affected by dataset shifts, improving detection accuracy and providing actionable insights.

Contribution

The paper presents a novel explainable shift detection method that leverages concept bottlenecks to identify and rank affected concepts, enhancing interpretability and accuracy.

Findings

CBSD accurately detects concepts affected by dataset shifts.

CBSD outperforms state-of-the-art shift detection methods.

Case studies demonstrate CBSD's effectiveness on dSprites and 3dshapes.

Abstract

Despite their remarkable performance on a wide range of visual tasks, machine learning technologies often succumb to data distribution shifts. Consequently, a range of recent work explores techniques for detecting these shifts. Unfortunately, current techniques offer no explanations about what triggers the detection of shifts, thus limiting their utility to provide actionable insights. In this work, we present Concept Bottleneck Shift Detection (CBSD): a novel explainable shift detection method. CBSD provides explanations by identifying and ranking the degree to which high-level human-understandable concepts are affected by shifts. Using two case studies (dSprites and 3dshapes), we demonstrate how CBSD can accurately detect underlying concepts that are affected by shifts and achieve higher detection accuracy compared to state-of-the-art shift detection methods.