Boosting Video Super Resolution with Patch-Based Temporal Redundancy Optimization

TL;DR

This paper introduces methods to handle temporal redundancy in video super-resolution, improving existing algorithms' performance on diverse real-world videos without degrading their results on standard datasets.

Contribution

It proposes two plug-and-play techniques to optimize patch-based temporal redundancy handling in VSR algorithms, enhancing robustness and performance in wild scenarios.

Findings

Significant performance improvement on wild videos

Maintains accuracy on standard datasets

Effective handling of stationary object redundancy

Abstract

The success of existing video super-resolution (VSR) algorithms stems mainly exploiting the temporal information from the neighboring frames. However, none of these methods have discussed the influence of the temporal redundancy in the patches with stationary objects and background and usually use all the information in the adjacent frames without any discrimination. In this paper, we observe that the temporal redundancy will bring adverse effect to the information propagation,which limits the performance of the most existing VSR methods. Motivated by this observation, we aim to improve existing VSR algorithms by handling the temporal redundancy patches in an optimized manner. We develop two simple yet effective plug and play methods to improve the performance of existing local and non-local propagation-based VSR algorithms on widely-used public videos. For more comprehensive evaluating the robustness and performance of existing VSR algorithms, we also collect a new dataset which contains a variety of public videos as testing set. Extensive evaluations show that the proposed methods can significantly improve the performance of existing VSR methods on the collected videos from wild scenarios while maintain their performance on existing commonly used datasets. The code is available at https://github.com/HYHsimon/Boosted-VSR.