Compressing Deep CNNs using Basis Representation and Spectral Fine-tuning

TL;DR

This paper introduces a simple and effective method for compressing deep CNNs by representing convolutional layers with basis filters and fine-tuning in the basis space, reducing model size while maintaining performance.

Contribution

It presents a novel basis representation approach for CNN compression, enabling direct optimization in basis space and demonstrating broad applicability across architectures and tasks.

Findings

Significant reduction in model size with minimal accuracy loss.

Effective compression demonstrated on multiple CNN architectures.

Reduced execution time and power consumption on embedded hardware.

Abstract

We propose an efficient and straightforward method for compressing deep convolutional neural networks (CNNs) that uses basis filters to represent the convolutional layers, and optimizes the performance of the compressed network directly in the basis space. Specifically, any spatial convolution layer of the CNN can be replaced by two successive convolution layers: the first is a set of three-dimensional orthonormal basis filters, followed by a layer of one-dimensional filters that represents the original spatial filters in the basis space. We jointly fine-tune both the basis and the filter representation to directly mitigate any performance loss due to the truncation. Generality of the proposed approach is demonstrated by applying it to several well known deep CNN architectures and data sets for image classification and object detection. We also present the execution time and power usage at different compression levels on the Xavier Jetson AGX processor.