Frames2Residual: Spatiotemporal Decoupling for Self-Supervised Video Denoising

TL;DR

Frames2Residual introduces a novel spatiotemporal decoupling framework for self-supervised video denoising, effectively separating temporal consistency modeling from spatial texture recovery to improve performance.

Contribution

The paper proposes a two-stage decoupling approach that explicitly separates temporal and spatial denoising processes, enhancing self-supervised video denoising performance.

Findings

Outperforms existing self-supervised methods on sRGB and raw video benchmarks.

Effectively models inter-frame temporal consistency while preserving intra-frame spatial details.

Demonstrates significant improvements through extensive experiments.

Abstract

Self-supervised video denoising methods typically extend image-based frameworks into the temporal dimension, yet they often struggle to integrate inter-frame temporal consistency with intra-frame spatial specificity. Existing Video Blind-Spot Networks (BSNs) require noise independence by masking the center pixel, this constraint prevents the use of spatial evidence for texture recovery, thereby severing spatiotemporal correlations and causing texture loss. To address this, we propose Frames2Residual (F2R), a spatiotemporal decoupling framework that explicitly divides self-supervised training into two distinct stages: blind temporal consistency modeling and non-blind spatial texture recovery. In Stage 1, a blind temporal estimator learns inter-frame consistency using a frame-wise blind strategy, producing a temporally consistent anchor. In Stage 2, a non-blind spatial refiner leverages this anchor to safely reintroduce the center frame and recover intra-frame high-frequency spatial residuals while preserving temporal stability. Extensive experiments demonstrate that our decoupling strategy allows F2R to outperform existing self-supervised methods on both sRGB and raw video benchmarks.