A Variational Information Bottleneck Based Method to Compress Sequential Networks for Human Action Recognition

TL;DR

This paper introduces a novel compression method for RNNs used in human action recognition, leveraging Variational Information Bottleneck theory and group-lasso regularization to significantly reduce model size while maintaining accuracy.

Contribution

It proposes a VIB-based pruning approach combined with group-lasso regularization for effective RNN compression in HAR tasks, achieving unprecedented compression ratios.

Findings

Over 70x compression on UCF11 dataset

Maintains accuracy with significantly reduced model size

Increases inference speed multiple times

Abstract

In the last few years, compression of deep neural networks has become an important strand of machine learning and computer vision research. Deep models require sizeable computational complexity and storage, when used for instance for Human Action Recognition (HAR) from videos, making them unsuitable to be deployed on edge devices. In this paper, we address this issue and propose a method to effectively compress Recurrent Neural Networks (RNNs) such as Gated Recurrent Units (GRUs) and Long-Short-Term-Memory Units (LSTMs) that are used for HAR. We use a Variational Information Bottleneck (VIB) theory-based pruning approach to limit the information flow through the sequential cells of RNNs to a small subset. Further, we combine our pruning method with a specific group-lasso regularization technique that significantly improves compression. The proposed techniques reduce model parameters and memory footprint from latent representations, with little or no reduction in the validation accuracy while increasing the inference speed several-fold. We perform experiments on the three widely used Action Recognition datasets, viz. UCF11, HMDB51, and UCF101, to validate our approach. It is shown that our method achieves over 70 times greater compression than the nearest competitor with comparable accuracy for the task of action recognition on UCF11.