Decomposing the Deep: Finding Class Specific Filters in Deep CNNs

TL;DR

This paper introduces a method to interpret deep CNNs by identifying class-specific features in the final layers, revealing a low-dimensional decision surface and improving interpretability and efficiency.

Contribution

It presents an efficient approach to decompose CNN decision layers into interpretable subspaces, highlighting the low-dimensional nature of class decisions.

Findings

Number of class-specific features is much lower than layer dimension

Decision surface lies on a low-dimensional manifold

Decomposition improves interpretability and reduces computation

Abstract

Interpretability of Deep Neural Networks has become a major area of exploration. Although these networks have achieved state of the art accuracy in many tasks, it is extremely difficult to interpret and explain their decisions. In this work we analyze the final and penultimate layers of Deep Convolutional Networks and provide an efficient method for identifying subsets of features that contribute most towards the network's decision for a class. We demonstrate that the number of such features per class is much lower in comparison to the dimension of the final layer and therefore the decision surface of Deep CNNs lies on a low dimensional manifold and is proportional to the network depth. Our methods allow to decompose the final layer into separate subspaces which is far more interpretable and has a lower computational cost as compared to the final layer of the full network.