Approximating Continuous Convolutions for Deep Network Compression

TL;DR

ApproxConv introduces a continuous convolution approach for CNN compression, effectively reducing model size by half with minimal accuracy loss and compatibility with other compression techniques.

Contribution

It proposes a novel continuous convolution framework that captures CNN filter structures with fewer parameters, enabling efficient compression with minimal fine-tuning.

Findings

Compressed CNNs by 50% with only 1.86% accuracy loss.

Compatible with quantisation for further size reduction.

Requires minimal fine-tuning after compression.

Abstract

We present ApproxConv, a novel method for compressing the layers of a convolutional neural network. Reframing conventional discrete convolution as continuous convolution of parametrised functions over space, we use functional approximations to capture the essential structures of CNN filters with fewer parameters than conventional operations. Our method is able to reduce the size of trained CNN layers requiring only a small amount of fine-tuning. We show that our method is able to compress existing deep network models by half whilst losing only 1.86% accuracy. Further, we demonstrate that our method is compatible with other compression methods like quantisation allowing for further reductions in model size.