Compressing complex convolutional neural network based on an improved deep compression algorithm

TL;DR

This paper extends deep compression techniques to complex-value CNNs, achieving significant model size reduction with minimal accuracy loss, thus enabling deployment on resource-constrained devices.

Contribution

It introduces an improved deep compression algorithm specifically for complex CNNs, a novel approach compared to prior real-valued network compression methods.

Findings

8x compression on CIFAR-10 with <3% accuracy loss

16x compression on ImageNet with ~2% accuracy loss

First to successfully compress complex CNNs at scale

Abstract

Although convolutional neural network (CNN) has made great progress, large redundant parameters restrict its deployment on embedded devices, especially mobile devices. The recent compression works are focused on real-value convolutional neural network (Real CNN), however, to our knowledge, there is no attempt for the compression of complex-value convolutional neural network (Complex CNN). Compared with the real-valued network, the complex-value neural network is easier to optimize, generalize, and has better learning potential. This paper extends the commonly used deep compression algorithm from real domain to complex domain and proposes an improved deep compression algorithm for the compression of Complex CNN. The proposed algorithm compresses the network about 8 times on CIFAR-10 dataset with less than 3% accuracy loss. On the ImageNet dataset, our method compresses the model about 16 times and the accuracy loss is about 2% without retraining.