The Unreasonable Effectiveness of Texture Transfer for Single Image Super-resolution

TL;DR

This paper demonstrates that simple texture loss alone can produce high-quality single image super-resolution results, outperforming complex models by leveraging a novel texture constraining approach and perceptual metrics.

Contribution

The authors introduce a semantically guided texture constraining method that significantly improves super-resolution quality using only texture loss, challenging the necessity of complex generative models.

Findings

Texture loss alone yields high perceptual quality images.

Texture representation of deep features better captures perceptual quality.

Off-the-shelf deep classifiers match calibrated perceptual metrics in quality assessment.

Abstract

While implicit generative models such as GANs have shown impressive results in high quality image reconstruction and manipulation using a combination of various losses, we consider a simpler approach leading to surprisingly strong results. We show that texture loss alone allows the generation of perceptually high quality images. We provide a better understanding of texture constraining mechanism and develop a novel semantically guided texture constraining method for further improvement. Using a recently developed perceptual metric employing "deep features" and termed LPIPS, the method obtains state-of-the-art results. Moreover, we show that a texture representation of those deep features better capture the perceptual quality of an image than the original deep features. Using texture information, off-the-shelf deep classification networks (without training) perform as well as the best performing (tuned and calibrated) LPIPS metrics. The code is publicly available.