Real-time Streaming Video Denoising with Bidirectional Buffers

TL;DR

This paper introduces a real-time streaming video denoising framework that leverages bidirectional temporal information through a novel buffer block, achieving high-quality denoising with efficient online inference.

Contribution

The paper proposes a Bidirectional Buffer Block enabling online bidirectional temporal fusion for streaming video denoising, improving fidelity and efficiency over existing methods.

Findings

Outperforms state-of-the-art models in fidelity and runtime

Effective for both synthetic and real noise in streaming videos

Applicable to both non-blind and blind denoising scenarios

Abstract

Video streams are delivered continuously to save the cost of storage and device memory. Real-time denoising algorithms are typically adopted on the user device to remove the noise involved during the shooting and transmission of video streams. However, sliding-window-based methods feed multiple input frames for a single output and lack computation efficiency. Recent multi-output inference works propagate the bidirectional temporal feature with a parallel or recurrent framework, which either suffers from performance drops on the temporal edges of clips or can not achieve online inference. In this paper, we propose a Bidirectional Streaming Video Denoising (BSVD) framework, to achieve high-fidelity real-time denoising for streaming videos with both past and future temporal receptive fields. The bidirectional temporal fusion for online inference is considered not applicable in the MoViNet. However, we introduce a novel Bidirectional Buffer Block as the core module of our BSVD, which makes it possible during our pipeline-style inference. In addition, our method is concise and flexible to be utilized in both non-blind and blind video denoising. We compare our model with various state-of-the-art video denoising models qualitatively and quantitatively on synthetic and real noise. Our method outperforms previous methods in terms of restoration fidelity and runtime. Our source code is publicly available at https://github.com/ChenyangQiQi/BSVD