A Deep Journey into Super-resolution: A survey

TL;DR

This survey comprehensively reviews over 30 deep learning-based super-resolution CNNs, comparing their architectures, performance, and challenges across multiple datasets, highlighting rapid progress and future research directions.

Contribution

It introduces a taxonomy of super-resolution networks, benchmarks numerous models, and analyzes architectural and performance differences in depth.

Findings

Deep learning models have significantly improved super-resolution accuracy.

Current models are more complex and require more resources.

Benchmark methods have been surpassed by recent approaches.

Abstract

Deep convolutional networks based super-resolution is a fast-growing field with numerous practical applications. In this exposition, we extensively compare 30+ state-of-the-art super-resolution Convolutional Neural Networks (CNNs) over three classical and three recently introduced challenging datasets to benchmark single image super-resolution. We introduce a taxonomy for deep-learning based super-resolution networks that groups existing methods into nine categories including linear, residual, multi-branch, recursive, progressive, attention-based and adversarial designs. We also provide comparisons between the models in terms of network complexity, memory footprint, model input and output, learning details, the type of network losses and important architectural differences (e.g., depth, skip-connections, filters). The extensive evaluation performed, shows the consistent and rapid growth in the accuracy in the past few years along with a corresponding boost in model complexity and the availability of large-scale datasets. It is also observed that the pioneering methods identified as the benchmark have been significantly outperformed by the current contenders. Despite the progress in recent years, we identify several shortcomings of existing techniques and provide future research directions towards the solution of these open problems.