VSpSR: Explorable Super-Resolution via Variational Sparse Representation

TL;DR

VSpSR introduces a variational sparse representation framework for super-resolution, enabling exploration of diverse high-resolution images from a single low-resolution input by sampling sparse coefficients.

Contribution

The paper develops a novel variational sparse framework that models the stochastic one-to-many SR mapping using neural networks and sparse representations.

Findings

Achieved diverse HR image generation through sampling sparse coefficients.

Ranked 7th in NTIRE 2021 SR challenge based on preliminary results.

Implementation is publicly available at the provided URL.

Abstract

Super-resolution (SR) is an ill-posed problem, which means that infinitely many high-resolution (HR) images can be degraded to the same low-resolution (LR) image. To study the one-to-many stochastic SR mapping, we implicitly represent the non-local self-similarity of natural images and develop a Variational Sparse framework for Super-Resolution (VSpSR) via neural networks. Since every small patch of a HR image can be well approximated by the sparse representation of atoms in an over-complete dictionary, we design a two-branch module, i.e., VSpM, to explore the SR space. Concretely, one branch of VSpM extracts patch-level basis from the LR input, and the other branch infers pixel-wise variational distributions with respect to the sparse coefficients. By repeatedly sampling coefficients, we could obtain infinite sparse representations, and thus generate diverse HR images. According to the preliminary results of NTIRE 2021 challenge on learning SR space, our team (FudanZmic21) ranks 7-th in terms of released scores. The implementation of VSpSR is released at https://zmiclab.github.io/.