Large Motion Video Super-Resolution with Dual Subnet and Multi-Stage Communicated Upsampling

TL;DR

This paper introduces a novel deep neural network for video super-resolution that effectively handles large motion by utilizing a dual subnet, multi-stage upsampling, and advanced motion estimation techniques, resulting in superior performance.

Contribution

The paper proposes DSMC, a new neural network with U3D-RDN and MSCU modules, enhancing large-motion video super-resolution with improved motion estimation and training strategies.

Findings

Outperforms state-of-the-art methods on large-motion videos.

Effective motion compensation via U3D-RDN module.

Dual subnet improves training stability and generalization.

Abstract

Video super-resolution (VSR) aims at restoring a video in low-resolution (LR) and improving it to higher-resolution (HR). Due to the characteristics of video tasks, it is very important that motion information among frames should be well concerned, summarized and utilized for guidance in a VSR algorithm. Especially, when a video contains large motion, conventional methods easily bring incoherent results or artifacts. In this paper, we propose a novel deep neural network with Dual Subnet and Multi-stage Communicated Upsampling (DSMC) for super-resolution of videos with large motion. We design a new module named U-shaped residual dense network with 3D convolution (U3D-RDN) for fine implicit motion estimation and motion compensation (MEMC) as well as coarse spatial feature extraction. And we present a new Multi-Stage Communicated Upsampling (MSCU) module to make full use of the intermediate results of upsampling for guiding the VSR. Moreover, a novel dual subnet is devised to aid the training of our DSMC, whose dual loss helps to reduce the solution space as well as enhance the generalization ability. Our experimental results confirm that our method achieves superior performance on videos with large motion compared to state-of-the-art methods.