BRIEF: Backward Reduction of CNNs with Information Flow Analysis

TL;DR

BRIEF is a novel backward reduction algorithm for CNNs that leverages information flow analysis to significantly reduce model size while maintaining performance, outperforming previous methods especially on ResNet-34, SqueezeNet, and MobileNet.

Contribution

This paper introduces BRIEF, a backward reduction algorithm based on information flow analysis that effectively prunes redundant neural channels in CNNs, achieving superior model compression.

Findings

ResNet-34 model reduced by 32.3% on ImageNet, 3X better than previous methods.

Achieved additional reductions of 10.81% for SqueezeNet and 37.56% for MobileNet.

Model performance degradation is negligible despite significant size reduction.

Abstract

This paper proposes BRIEF, a backward reduction algorithm that explores compact CNN-model designs from the information flow perspective. This algorithm can remove substantial non-zero weighting parameters (redundant neural channels) of a network by considering its dynamic behavior, which traditional model-compaction techniques cannot achieve. With the aid of our proposed algorithm, we achieve significant model reduction on ResNet-34 in the ImageNet scale (32.3% reduction), which is 3X better than the previous result (10.8%). Even for highly optimized models such as SqueezeNet and MobileNet, we can achieve additional 10.81% and 37.56% reduction, respectively, with negligible performance degradation.