Compressing CNN Kernels for Videos Using Tucker Decompositions: Towards Lightweight CNN Applications

TL;DR

This paper extends Tucker decomposition-based CNN kernel compression from images to videos, achieving significant memory reduction but limited computational speed-up, facilitating lightweight video analysis on resource-constrained devices.

Contribution

It generalizes Tucker decomposition for 3D video data and evaluates its effectiveness in reducing memory and computation in CNNs for videos.

Findings

Memory compression factor of 51

Computational speed-up of 1.4 (less than expected)

Maintains comparable accuracy to original networks

Abstract

Convolutional Neural Networks (CNN) are the state-of-the-art in the field of visual computing. However, a major problem with CNNs is the large number of floating point operations (FLOPs) required to perform convolutions for large inputs. When considering the application of CNNs to video data, convolutional filters become even more complex due to the extra temporal dimension. This leads to problems when respective applications are to be deployed on mobile devices, such as smart phones, tablets, micro-controllers or similar, indicating less computational power. Kim et al. (2016) proposed using a Tucker-decomposition to compress the convolutional kernel of a pre-trained network for images in order to reduce the complexity of the network, i.e. the number of FLOPs. In this paper, we generalize the aforementioned method for application to videos (and other 3D signals) and evaluate the proposed method on a modified version of the THETIS data set, which contains videos of individuals performing tennis shots. We show that the compressed network reaches comparable accuracy, while indicating a memory compression by a factor of 51. However, the actual computational speed-up (factor 1.4) does not meet our theoretically derived expectation (factor 6).