Journey Towards Tiny Perceptual Super-Resolution

TL;DR

This paper introduces a neural architecture search method to develop tiny perceptual super-resolution models that outperform existing models in quality while being significantly more efficient for on-device deployment.

Contribution

The work presents a novel NAS approach that jointly searches generator and discriminator architectures for small perceptual SR models, enabling efficient on-device super-resolution.

Findings

Outperforms SRGAN and EnhanceNet in perceptual quality metrics

Achieves up to 26.4× memory efficiency and 33.6× compute efficiency

Demonstrates effective search for SR-optimized discriminator architectures

Abstract

Recent works in single-image perceptual super-resolution (SR) have demonstrated unprecedented performance in generating realistic textures by means of deep convolutional networks. However, these convolutional models are excessively large and expensive, hindering their effective deployment to end devices. In this work, we propose a neural architecture search (NAS) approach that integrates NAS and generative adversarial networks (GANs) with recent advances in perceptual SR and pushes the efficiency of small perceptual SR models to facilitate on-device execution. Specifically, we search over the architectures of both the generator and the discriminator sequentially, highlighting the unique challenges and key observations of searching for an SR-optimized discriminator and comparing them with existing discriminator architectures in the literature. Our tiny perceptual SR (TPSR) models outperform SRGAN and EnhanceNet on both full-reference perceptual metric (LPIPS) and distortion metric (PSNR) while being up to 26.4$\times$ more memory efficient and 33.6$\times$ more compute efficient respectively.