Perceptual Image Enhancement for Smartphone Real-Time Applications

TL;DR

This paper introduces LPIENet, a lightweight deep learning model designed for real-time perceptual image enhancement on smartphones, effectively removing common artifacts with minimal computational resources.

Contribution

The paper presents a novel lightweight neural network architecture optimized for real-time image enhancement on mobile devices, balancing performance and efficiency.

Findings

LPIENet achieves competitive artifact removal performance with fewer parameters.

The model processes 2K images under 1 second on mid-level smartphones.

Deployment on commercial smartphones demonstrates practical real-time capability.

Abstract

Recent advances in camera designs and imaging pipelines allow us to capture high-quality images using smartphones. However, due to the small size and lens limitations of the smartphone cameras, we commonly find artifacts or degradation in the processed images. The most common unpleasant effects are noise artifacts, diffraction artifacts, blur, and HDR overexposure. Deep learning methods for image restoration can successfully remove these artifacts. However, most approaches are not suitable for real-time applications on mobile devices due to their heavy computation and memory requirements. In this paper, we propose LPIENet, a lightweight network for perceptual image enhancement, with the focus on deploying it on smartphones. Our experiments show that, with much fewer parameters and operations, our model can deal with the mentioned artifacts and achieve competitive performance compared with state-of-the-art methods on standard benchmarks. Moreover, to prove the efficiency and reliability of our approach, we deployed the model directly on commercial smartphones and evaluated its performance. Our model can process 2K resolution images under 1 second in mid-level commercial smartphones.