Interpreting CNN Knowledge via an Explanatory Graph

TL;DR

This paper introduces an unsupervised method to interpret CNNs by constructing an explanatory graph that reveals hierarchical object part knowledge, improving part localization accuracy.

Contribution

It proposes a novel unsupervised approach to disentangle object parts within CNN filters and build an explanatory graph for better interpretability.

Findings

Each graph node consistently represents the same object part across images.

The method significantly outperforms existing approaches in part localization.

The explanatory graph reveals the hierarchical structure of learned object parts.

Abstract

This paper learns a graphical model, namely an explanatory graph, which reveals the knowledge hierarchy hidden inside a pre-trained CNN. Considering that each filter in a conv-layer of a pre-trained CNN usually represents a mixture of object parts, we propose a simple yet efficient method to automatically disentangles different part patterns from each filter, and construct an explanatory graph. In the explanatory graph, each node represents a part pattern, and each edge encodes co-activation relationships and spatial relationships between patterns. More importantly, we learn the explanatory graph for a pre-trained CNN in an unsupervised manner, i.e., without a need of annotating object parts. Experiments show that each graph node consistently represents the same object part through different images. We transfer part patterns in the explanatory graph to the task of part localization, and our method significantly outperforms other approaches.