Patch Craft: Video Denoising by Deep Modeling and Patch Matching

TL;DR

This paper introduces a novel video denoising method that enhances CNN performance by augmenting videos with artificial patch-craft frames, effectively leveraging self-similarity for improved results.

Contribution

It proposes the innovative concept of patch-craft frames to incorporate self-similarity explicitly into CNN-based video denoising.

Findings

Significant improvement in denoising quality using patch-craft frames.

Effective utilization of self-similarity in CNN-based video denoising.

Enhanced performance over traditional methods without explicit self-similarity modeling.

Abstract

The non-local self-similarity property of natural images has been exploited extensively for solving various image processing problems. When it comes to video sequences, harnessing this force is even more beneficial due to the temporal redundancy. In the context of image and video denoising, many classically-oriented algorithms employ self-similarity, splitting the data into overlapping patches, gathering groups of similar ones and processing these together somehow. With the emergence of convolutional neural networks (CNN), the patch-based framework has been abandoned. Most CNN denoisers operate on the whole image, leveraging non-local relations only implicitly by using a large receptive field. This work proposes a novel approach for leveraging self-similarity in the context of video denoising, while still relying on a regular convolutional architecture. We introduce a concept of patch-craft frames - artificial frames that are similar to the real ones, built by tiling matched patches. Our algorithm augments video sequences with patch-craft frames and feeds them to a CNN. We demonstrate the substantial boost in denoising performance obtained with the proposed approach.