Video Super Resolution Based on Deep Learning: A Comprehensive Survey

TL;DR

This comprehensive survey reviews 33 deep learning-based video super-resolution methods, classifying them into six categories, detailing architectures, and comparing their performance on benchmark datasets to advance understanding in the field.

Contribution

First systematic review of deep learning-based VSR methods, providing a taxonomy, detailed analysis, and performance comparison to guide future research.

Findings

Deep learning significantly advances VSR performance.

Methods utilize inter-frame information in six distinct ways.

Benchmark comparisons highlight leading approaches.

Abstract

In recent years, deep learning has made great progress in many fields such as image recognition, natural language processing, speech recognition and video super-resolution. In this survey, we comprehensively investigate 33 state-of-the-art video super-resolution (VSR) methods based on deep learning. It is well known that the leverage of information within video frames is important for video super-resolution. Thus we propose a taxonomy and classify the methods into six sub-categories according to the ways of utilizing inter-frame information. Moreover, the architectures and implementation details of all the methods are depicted in detail. Finally, we summarize and compare the performance of the representative VSR method on some benchmark datasets. We also discuss some challenges, which need to be further addressed by researchers in the community of VSR. To the best of our knowledge, this work is the first systematic review on VSR tasks, and it is expected to make a contribution to the development of recent studies in this area and potentially deepen our understanding to the VSR techniques based on deep learning.