Composite Binary Decomposition Networks

TL;DR

CBDNet introduces a composite binary decomposition approach that reduces parameters and computations in neural networks by composing and decomposing binary tensors, maintaining accuracy while improving efficiency.

Contribution

The paper proposes a novel composite binary decomposition method that significantly reduces parameters and operations in neural networks with minimal accuracy loss.

Findings

CBDNet can approximate ResNet-18 with 5.25 bits and minor accuracy drop.

CBDNet achieves similar efficiency for VGG-16, DenseNet-121, SSD300, and SegNet.

The method maintains high accuracy while greatly reducing resource requirements.

Abstract

Binary neural networks have great resource and computing efficiency, while suffer from long training procedure and non-negligible accuracy drops, when comparing to the full-precision counterparts. In this paper, we propose the composite binary decomposition networks (CBDNet), which first compose real-valued tensor of each layer with a limited number of binary tensors, and then decompose some conditioned binary tensors into two low-rank binary tensors, so that the number of parameters and operations are greatly reduced comparing to the original ones. Experiments demonstrate the effectiveness of the proposed method, as CBDNet can approximate image classification network ResNet-18 using 5.25 bits, VGG-16 using 5.47 bits, DenseNet-121 using 5.72 bits, object detection networks SSD300 using 4.38 bits, and semantic segmentation networks SegNet using 5.18 bits, all with minor accuracy drops.