Unidirectional Video Denoising by Mimicking Backward Recurrent Modules with Look-ahead Forward Ones

TL;DR

This paper introduces a unidirectional video denoising network that mimics bidirectional recurrent modules using look-ahead forward modules, achieving state-of-the-art results with constant latency and memory efficiency.

Contribution

It proposes a novel recurrent network with forward and look-ahead modules for real-time video denoising, overcoming the offline limitations of bidirectional RNNs.

Findings

Achieves state-of-the-art denoising performance.

Maintains constant latency and memory usage.

Effectively leverages near-future frames for improved denoising.

Abstract

While significant progress has been made in deep video denoising, it remains very challenging for exploiting historical and future frames. Bidirectional recurrent networks (BiRNN) have exhibited appealing performance in several video restoration tasks. However, BiRNN is intrinsically offline because it uses backward recurrent modules to propagate from the last to current frames, which causes high latency and large memory consumption. To address the offline issue of BiRNN, we present a novel recurrent network consisting of forward and look-ahead recurrent modules for unidirectional video denoising. Particularly, look-ahead module is an elaborate forward module for leveraging information from near-future frames. When denoising the current frame, the hidden features by forward and look-ahead recurrent modules are combined, thereby making it feasible to exploit both historical and near-future frames. Due to the scene motion between non-neighboring frames, border pixels missing may occur when warping look-ahead feature from near-future frame to current frame, which can be largely alleviated by incorporating forward warping and proposed border enlargement. Experiments show that our method achieves state-of-the-art performance with constant latency and memory consumption. Code is avaliable at https://github.com/nagejacob/FloRNN.