Understanding Failures of Deep Networks via Robust Feature Extraction

TL;DR

This paper presents a novel method for identifying and visualizing failure modes in deep networks by leveraging robust features, aiding understanding and debugging beyond traditional aggregate metrics.

Contribution

It introduces a feature-based failure analysis approach that does not rely on crowdsourced labels and includes visualization tools for interpretability.

Findings

Effective discovery of failure modes on ImageNet

Visualization aids human understanding of features

Insights assist engineers in debugging models

Abstract

Traditional evaluation metrics for learned models that report aggregate scores over a test set are insufficient for surfacing important and informative patterns of failure over features and instances. We introduce and study a method aimed at characterizing and explaining failures by identifying visual attributes whose presence or absence results in poor performance. In distinction to previous work that relies upon crowdsourced labels for visual attributes, we leverage the representation of a separate robust model to extract interpretable features and then harness these features to identify failure modes. We further propose a visualization method aimed at enabling humans to understand the meaning encoded in such features and we test the comprehensibility of the features. An evaluation of the methods on the ImageNet dataset demonstrates that: (i) the proposed workflow is effective for discovering important failure modes, (ii) the visualization techniques help humans to understand the extracted features, and (iii) the extracted insights can assist engineers with error analysis and debugging.