Lightweight Modules for Efficient Deep Learning based Image Restoration

TL;DR

This paper introduces lightweight modules for deep learning-based image restoration, reducing computational costs while maintaining high-quality results, specifically tailored for resource-constrained devices like smartphones.

Contribution

It designs and analyzes generic efficient modules for image-to-image translation tasks, addressing the limitations of existing classification-focused efficient network methods.

Findings

Proposed modules reduce parameters and memory footprint significantly.

Networks achieve comparable visual quality to full models.

Analysis shows depthwise separable convolutions may not suit all low-level vision tasks.

Abstract

Low level image restoration is an integral component of modern artificial intelligence (AI) driven camera pipelines. Most of these frameworks are based on deep neural networks which present a massive computational overhead on resource constrained platform like a mobile phone. In this paper, we propose several lightweight low-level modules which can be used to create a computationally low cost variant of a given baseline model. Recent works for efficient neural networks design have mainly focused on classification. However, low-level image processing falls under the image-to-image' translation genre which requires some additional computational modules not present in classification. This paper seeks to bridge this gap by designing generic efficient modules which can replace essential components used in contemporary deep learning based image restoration networks. We also present and analyse our results highlighting the drawbacks of applying depthwise separable convolutional kernel (a popular method for efficient classification network) for sub-pixel convolution based upsampling (a popular upsampling strategy for low-level vision applications). This shows that concepts from domain of classification cannot always be seamlessly integrated into image-to-image translation tasks. We extensively validate our findings on three popular tasks of image inpainting, denoising and super-resolution. Our results show that proposed networks consistently output visually similar reconstructions compared to full capacity baselines with significant reduction of parameters, memory footprint and execution speeds on contemporary mobile devices.