Orthogonally Regularized Deep Networks For Image Super-resolution

TL;DR

This paper introduces a novel deep learning architecture for image super-resolution that operates in the transform domain, specifically using a trainable and orthogonally regularized DCT layer to improve efficiency and adaptivity.

Contribution

It proposes a new network structure that integrates a trainable, orthogonally constrained DCT layer for super-resolution, enhancing efficiency and adaptability over spatial domain methods.

Findings

The DCT layer can be integrated and made trainable within the network.

Orthogonality constraints improve the transform basis adaptation.

The method simplifies super-resolution by leveraging the transform domain.

Abstract

Deep learning methods, in particular trained Convolutional Neural Networks (CNNs) have recently been shown to produce compelling state-of-the-art results for single image Super-Resolution (SR). Invariably, a CNN is learned to map the low resolution (LR) image to its corresponding high resolution (HR) version in the spatial domain. Aiming for faster inference and more efficient solutions than solving the SR problem in the spatial domain, we propose a novel network structure for learning the SR mapping function in an image transform domain, specifically the Discrete Cosine Transform (DCT). As a first contribution, we show that DCT can be integrated into the network structure as a Convolutional DCT (CDCT) layer. We further extend the network to allow the CDCT layer to become trainable (i.e. optimizable). Because this layer represents an image transform, we enforce pairwise orthogonality constraints on the individual basis functions/filters. This Orthogonally Regularized Deep SR network (ORDSR) simplifies the SR task by taking advantage of image transform domain while adapting the design of transform basis to the training image set.