Joint Video Enhancement with Deblurring, Super-Resolution, and Frame Interpolation Network

TL;DR

This paper introduces DSFN, a unified neural network that simultaneously enhances video resolution, frame rate, and clarity by integrating deblurring, super-resolution, and frame interpolation, outperforming sequential methods.

Contribution

The paper presents a novel joint video enhancement network that addresses multiple degradation factors simultaneously, improving efficiency and performance over traditional sequential approaches.

Findings

Outperforms state-of-the-art sequential methods on public datasets.

Achieves higher quality video with smaller network size.

Faster processing time than existing techniques.

Abstract

Video quality is often severely degraded by multiple factors rather than a single factor. These low-quality videos can be restored to high-quality videos by sequentially performing appropriate video enhancement techniques. However, the sequential approach was inefficient and sub-optimal because most video enhancement approaches were designed without taking into account that multiple factors together degrade video quality. In this paper, we propose a new joint video enhancement method that mitigates multiple degradation factors simultaneously by resolving an integrated enhancement problem. Our proposed network, named DSFN, directly produces a high-resolution, high-frame-rate, and clear video from a low-resolution, low-frame-rate, and blurry video. In the DSFN, low-resolution and blurry input frames are enhanced by a joint deblurring and super-resolution (JDSR) module. Meanwhile, intermediate frames between input adjacent frames are interpolated by a triple-frame-based frame interpolation (TFBFI) module. The proper combination of the proposed modules of DSFN can achieve superior performance on the joint video enhancement task. Experimental results show that the proposed method outperforms other sequential state-of-the-art techniques on public datasets with a smaller network size and faster processing time.