Learning Graph Regularisation for Guided Super-Resolution

TL;DR

This paper presents a novel guided super-resolution method using a differentiable graph optimisation layer that leverages rich contextual information and guarantees high fidelity to the source, resulting in sharper, more natural images.

Contribution

It introduces a differentiable graph-based optimisation layer for guided super-resolution that explicitly enforces fidelity and learns rich context from guide images.

Findings

Outperforms recent baselines in reconstruction accuracy

Produces sharper, more natural-looking images

Generalizes well to unseen datasets

Abstract

We introduce a novel formulation for guided super-resolution. Its core is a differentiable optimisation layer that operates on a learned affinity graph. The learned graph potentials make it possible to leverage rich contextual information from the guide image, while the explicit graph optimisation within the architecture guarantees rigorous fidelity of the high-resolution target to the low-resolution source. With the decision to employ the source as a constraint rather than only as an input to the prediction, our method differs from state-of-the-art deep architectures for guided super-resolution, which produce targets that, when downsampled, will only approximately reproduce the source. This is not only theoretically appealing, but also produces crisper, more natural-looking images. A key property of our method is that, although the graph connectivity is restricted to the pixel lattice, the associated edge potentials are learned with a deep feature extractor and can encode rich context information over large receptive fields. By taking advantage of the sparse graph connectivity, it becomes possible to propagate gradients through the optimisation layer and learn the edge potentials from data. We extensively evaluate our method on several datasets, and consistently outperform recent baselines in terms of quantitative reconstruction errors, while also delivering visually sharper outputs. Moreover, we demonstrate that our method generalises particularly well to new datasets not seen during training.