Binary-decomposed DCNN for accelerating computation and compressing model without retraining

TL;DR

This paper introduces a binary decomposition method for DCNNs that accelerates inference and compresses models significantly without retraining, making deep learning more feasible on low-performance devices.

Contribution

The proposed Binary-decomposed DCNN replaces real-valued computations with binary ones, enabling faster inference and smaller models without retraining.

Findings

Speed increased by up to 2.07 times

Model size reduced by approximately 80%

Error rate increase limited to around 2.16%

Abstract

Recent trends show recognition accuracy increasing even more profoundly. Inference process of Deep Convolutional Neural Networks (DCNN) has a large number of parameters, requires a large amount of computation, and can be very slow. The large number of parameters also require large amounts of memory. This is resulting in increasingly long computation times and large model sizes. To implement mobile and other low performance devices incorporating DCNN, model sizes must be compressed and computation must be accelerated. To that end, this paper proposes Binary-decomposed DCNN, which resolves these issues without the need for retraining. Our method replaces real-valued inner-product computations with binary inner-product computations in existing network models to accelerate computation of inference and decrease model size without the need for retraining. Binary computations can be done at high speed using logical operators such as XOR and AND, together with bit counting. In tests using AlexNet with the ImageNet classification task, speed increased by a factor of 1.79, models were compressed by approximately 80%, and increase in error rate was limited to 1.20%. With VGG-16, speed increased by a factor of 2.07, model sizes decreased by 81%, and error increased by only 2.16%.