GLEAN: Generative Latent Bank for Image Super-Resolution and Beyond

TL;DR

GLEAN leverages pre-trained GANs as a latent prior bank to enhance image super-resolution and other restoration tasks efficiently, outperforming existing methods with fewer parameters and computations.

Contribution

The paper introduces GLEAN, a novel approach that directly uses pre-trained GANs as priors for super-resolution, avoiding costly optimization and enabling diverse applications.

Findings

GLEAN achieves high-quality super-resolution with a single forward pass.

LightGLEAN reduces parameters and FLOPs significantly while maintaining performance.

The method extends effectively to colorization and blind restoration tasks.

Abstract

We show that pre-trained Generative Adversarial Networks (GANs) such as StyleGAN and BigGAN can be used as a latent bank to improve the performance of image super-resolution. While most existing perceptual-oriented approaches attempt to generate realistic outputs through learning with adversarial loss, our method, Generative LatEnt bANk (GLEAN), goes beyond existing practices by directly leveraging rich and diverse priors encapsulated in a pre-trained GAN. But unlike prevalent GAN inversion methods that require expensive image-specific optimization at runtime, our approach only needs a single forward pass for restoration. GLEAN can be easily incorporated in a simple encoder-bank-decoder architecture with multi-resolution skip connections. Employing priors from different generative models allows GLEAN to be applied to diverse categories (\eg~human faces, cats, buildings, and cars). We further present a lightweight version of GLEAN, named LightGLEAN, which retains only the critical components in GLEAN. Notably, LightGLEAN consists of only 21% of parameters and 35% of FLOPs while achieving comparable image quality. We extend our method to different tasks including image colorization and blind image restoration, and extensive experiments show that our proposed models perform favorably in comparison to existing methods. Codes and models are available at https://github.com/open-mmlab/mmediting.