Structure-Preserving Image Super-resolution via Contextualized Multi-task Learning

TL;DR

This paper introduces a structure-preserving image super-resolution method using a multi-task learning framework that enhances high-frequency details by jointly predicting boundaries, residuals, and high-resolution images, leading to improved quality and efficiency.

Contribution

It proposes a novel multi-task learning approach that explicitly models structural details and residuals for better super-resolution results, outperforming existing CNN-based methods.

Findings

Achieves higher restoration quality on standard benchmarks.

Demonstrates improved computational efficiency.

Effectively preserves structural details in super-resolved images.

Abstract

Single image super resolution (SR), which refers to reconstruct a higher-resolution (HR) image from the observed low-resolution (LR) image, has received substantial attention due to its tremendous application potentials. Despite the breakthroughs of recently proposed SR methods using convolutional neural networks (CNNs), their generated results usually lack of preserving structural (high-frequency) details. In this paper, regarding global boundary context and residual context as complimentary information for enhancing structural details in image restoration, we develop a contextualized multi-task learning framework to address the SR problem. Specifically, our method first extracts convolutional features from the input LR image and applies one deconvolutional module to interpolate the LR feature maps in a content-adaptive way. Then, the resulting feature maps are fed into two branched sub-networks. During the neural network training, one sub-network outputs salient image boundaries and the HR image, and the other sub-network outputs the local residual map, i.e., the residual difference between the generated HR image and ground-truth image. On several standard benchmarks (i.e., Set5, Set14 and BSD200), our extensive evaluations demonstrate the effectiveness of our SR method on achieving both higher restoration quality and computational efficiency compared with several state-of-the-art SR approaches. The source code and some SR results can be found at: http://hcp.sysu.edu.cn/structure-preserving-image-super-resolution/