Finding Patterns in Ambiguity: Interpretable Stress Testing in the Decision~Boundary

TL;DR

This paper introduces a method to interpret deep binary classifiers by selecting representative boundary prototypes and applying explanation algorithms, improving understanding of low-confidence decisions.

Contribution

It proposes a novel approach combining prototype selection and post-model explanations to enhance interpretability of decision boundaries in deep classifiers.

Findings

Revealed distinct, compact clusters of prototypes.

Captured essential features leading to low-confidence decisions.

Demonstrated effectiveness through visualizations and GradientSHAP analysis.

Abstract

The increasing use of deep learning across various domains highlights the importance of understanding the decision-making processes of these black-box models. Recent research focusing on the decision boundaries of deep classifiers, relies on generated synthetic instances in areas of low confidence, uncovering samples that challenge both models and humans. We propose a novel approach to enhance the interpretability of deep binary classifiers by selecting representative samples from the decision boundary - prototypes - and applying post-model explanation algorithms. We evaluate the effectiveness of our approach through 2D visualizations and GradientSHAP analysis. Our experiments demonstrate the potential of the proposed method, revealing distinct and compact clusters and diverse prototypes that capture essential features that lead to low-confidence decisions. By offering a more aggregated view of deep classifiers' decision boundaries, our work contributes to the responsible development and deployment of reliable machine learning systems.