Deep Parametric 3D Filters for Joint Video Denoising and Illumination Enhancement in Video Super Resolution

TL;DR

This paper introduces Deep Parametric 3D Filters (DP3DF), a novel method that simultaneously denoises, enhances illumination, and super-resolves low-light, noisy videos efficiently within a single neural network, outperforming existing methods.

Contribution

The paper proposes DP3DF, a new parametric representation that integrates local spatiotemporal info for joint video denoising, illumination enhancement, and super-resolution in one model.

Findings

Outperforms state-of-the-art methods on real datasets.

Achieves higher PSNR and user ratings.

Operates with very fast runtime.

Abstract

Despite the quality improvement brought by the recent methods, video super-resolution (SR) is still very challenging, especially for videos that are low-light and noisy. The current best solution is to subsequently employ best models of video SR, denoising, and illumination enhancement, but doing so often lowers the image quality, due to the inconsistency between the models. This paper presents a new parametric representation called the Deep Parametric 3D Filters (DP3DF), which incorporates local spatiotemporal information to enable simultaneous denoising, illumination enhancement, and SR efficiently in a single encoder-and-decoder network. Also, a dynamic residual frame is jointly learned with the DP3DF via a shared backbone to further boost the SR quality. We performed extensive experiments, including a large-scale user study, to show our method's effectiveness. Our method consistently surpasses the best state-of-the-art methods on all the challenging real datasets with top PSNR and user ratings, yet having a very fast run time.