LPRNet: Lightweight Deep Network by Low-rank Pointwise Residual Convolution

TL;DR

LPRNet introduces a low-rank pointwise residual convolution to create a lightweight deep learning model that significantly reduces computational costs while maintaining competitive accuracy for visual recognition tasks.

Contribution

The paper proposes a novel low-rank pointwise residual convolution module, LPR, to enhance model compression and efficiency in lightweight neural networks.

Findings

LPRNet reduces Flops and memory usage significantly.

Achieves competitive accuracy on image classification and face alignment.

Effective replacement for modules in MobileNet and ShuffleNetv2.

Abstract

Deep learning has become popular in recent years primarily due to the powerful computing device such as GPUs. However, deploying these deep models to end-user devices, smart phones, or embedded systems with limited resources is challenging. To reduce the computation and memory costs, we propose a novel lightweight deep learning module by low-rank pointwise residual (LPR) convolution, called LPRNet. Essentially, LPR aims at using low-rank approximation in pointwise convolution to further reduce the module size, while keeping depthwise convolutions as the residual module to rectify the LPR module. This is critical when the low-rankness undermines the convolution process. We embody our design by replacing modules of identical input-output dimension in MobileNet and ShuffleNetv2. Experiments on visual recognition tasks including image classification and face alignment on popular benchmarks show that our LPRNet achieves competitive performance but with significant reduction of Flops and memory cost compared to the state-of-the-art deep models focusing on model compression.