Mobile-friendly Image de-noising: Hardware Conscious Optimization for Edge Application

TL;DR

This paper introduces a novel mobile-optimized image de-noising neural network designed with hardware-aware neural architecture search, achieving significant efficiency improvements while maintaining competitive accuracy on edge devices like smartphones.

Contribution

It presents the first hardware-aware NAS for a U-Net architecture tailored for mobile image de-noising, reducing parameters and latency with minimal accuracy loss.

Findings

12% fewer parameters compared to baseline models

~2-fold reduction in on-device latency on Samsung Galaxy S24 Ultra

Competitive accuracy with 18-fold reduction in GMACs compared to SOTA models

Abstract

Image enhancement is a critical task in computer vision and photography that is often entangled with noise. This renders the traditional Image Signal Processing (ISP) ineffective compared to the advances in deep learning. However, the success of such methods is increasingly associated with the ease of their deployment on edge devices, such as smartphones. This work presents a novel mobile-friendly network for image de-noising obtained with Entropy-Regularized differentiable Neural Architecture Search (NAS) on a hardware-aware search space for a U-Net architecture, which is first-of-its-kind. The designed model has 12% less parameters, with ~2-fold improvement in ondevice latency and 1.5-fold improvement in the memory footprint for a 0.7% drop in PSNR, when deployed and profiled on Samsung Galaxy S24 Ultra. Compared to the SOTA Swin-Transformer for Image Restoration, the proposed network had competitive accuracy with ~18-fold reduction in GMACs. Further, the network was tested successfully for Gaussian de-noising with 3 intensities on 4 benchmarks and real-world de-noising on 1 benchmark demonstrating its generalization ability.