Omniscient Video Super-Resolution

TL;DR

This paper introduces an omniscient framework for video super-resolution that utilizes past, present, and future SR outputs, surpassing existing methods in performance and visual quality.

Contribution

It proposes a novel omniscient framework that unifies and extends iterative, recurrent, and hybrid approaches for video super-resolution.

Findings

Outperforms state-of-the-art methods in objective metrics.

Produces superior subjective visual effects.

Demonstrates better complexity-performance trade-offs.

Abstract

Most recent video super-resolution (SR) methods either adopt an iterative manner to deal with low-resolution (LR) frames from a temporally sliding window, or leverage the previously estimated SR output to help reconstruct the current frame recurrently. A few studies try to combine these two structures to form a hybrid framework but have failed to give full play to it. In this paper, we propose an omniscient framework to not only utilize the preceding SR output, but also leverage the SR outputs from the present and future. The omniscient framework is more generic because the iterative, recurrent and hybrid frameworks can be regarded as its special cases. The proposed omniscient framework enables a generator to behave better than its counterparts under other frameworks. Abundant experiments on public datasets show that our method is superior to the state-of-the-art methods in objective metrics, subjective visual effects and complexity. Our code will be made public.