Interpreting CNNs via Decision Trees

TL;DR

This paper introduces a method to interpret CNN predictions by learning a decision tree that decomposes feature representations into object parts, providing semantic and quantitative explanations at multiple decision modes.

Contribution

It proposes a novel approach to explain CNN predictions by mapping feature activations to object parts using a decision tree, revealing decision modes and their contributions.

Findings

Effectively mines all potential decision modes of CNNs.

Provides semantic explanations beyond pixel-level analysis.

Organizes decision modes in a coarse-to-fine manner.

Abstract

This paper aims to quantitatively explain rationales of each prediction that is made by a pre-trained convolutional neural network (CNN). We propose to learn a decision tree, which clarifies the specific reason for each prediction made by the CNN at the semantic level. I.e., the decision tree decomposes feature representations in high conv-layers of the CNN into elementary concepts of object parts. In this way, the decision tree tells people which object parts activate which filters for the prediction and how much they contribute to the prediction score. Such semantic and quantitative explanations for CNN predictions have specific values beyond the traditional pixel-level analysis of CNNs. More specifically, our method mines all potential decision modes of the CNN, where each mode represents a common case of how the CNN uses object parts for prediction. The decision tree organizes all potential decision modes in a coarse-to-fine manner to explain CNN predictions at different fine-grained levels. Experiments have demonstrated the effectiveness of the proposed method.