A Main/Subsidiary Network Framework for Simplifying Binary Neural Network

TL;DR

This paper introduces a novel main/subsidiary network framework with a learning-based filter pruning method for binary neural networks, significantly reducing model size and improving efficiency while maintaining accuracy.

Contribution

It defines the filter-level pruning problem for binary neural networks and proposes a new learning-based approach with a layer-wise scheme, outperforming greedy methods.

Findings

Effective pruning of binary models like VGG-11 and ResNet-18

Significant reduction in memory and latency

Maintained or improved classification accuracy

Abstract

To reduce memory footprint and run-time latency, techniques such as neural network pruning and binarization have been explored separately. However, it is unclear how to combine the best of the two worlds to get extremely small and efficient models. In this paper, we, for the first time, define the filter-level pruning problem for binary neural networks, which cannot be solved by simply migrating existing structural pruning methods for full-precision models. A novel learning-based approach is proposed to prune filters in our main/subsidiary network framework, where the main network is responsible for learning representative features to optimize the prediction performance, and the subsidiary component works as a filter selector on the main network. To avoid gradient mismatch when training the subsidiary component, we propose a layer-wise and bottom-up scheme. We also provide the theoretical and experimental comparison between our learning-based and greedy rule-based methods. Finally, we empirically demonstrate the effectiveness of our approach applied on several binary models, including binarized NIN, VGG-11, and ResNet-18, on various image classification datasets.