DRNet: Dissect and Reconstruct the Convolutional Neural Network via Interpretable Manners

TL;DR

This paper introduces a method to identify and utilize only the most important channels in convolutional neural networks for specific tasks, reducing redundancy and improving efficiency without significant accuracy loss.

Contribution

It proposes an interpretable approach to dissect and reconstruct ConvNet channels tailored to individual classes, enhancing efficiency and interpretability.

Findings

Only 11% of parameters needed for two-class sub-tasks on CIFAR-10 with negligible accuracy loss.

Achieved an average 14.29% accuracy improvement on two-class sub-tasks for ImageNet.

Method captures semantic meanings of channels and targets context information more effectively.

Abstract

Convolutional neural networks (ConvNets) are widely used in real life. People usually use ConvNets which pre-trained on a fixed number of classes. However, for different application scenarios, we usually do not need all of the classes, which means ConvNets are redundant when dealing with these tasks. This paper focuses on the redundancy of ConvNet channels. We proposed a novel idea: using an interpretable manner to find the most important channels for every single class (dissect), and dynamically run channels according to classes in need (reconstruct). For VGG16 pre-trained on CIFAR-10, we only run 11\% parameters for two-classes sub-tasks on average with negligible accuracy loss. For VGG16 pre-trained on ImageNet, our method averagely gains 14.29\% accuracy promotion for two-classes sub-tasks. In addition, analysis show that our method captures some semantic meanings of channels, and uses the context information more targeted for sub-tasks of ConvNets.