Learning Raw Image Denoising with Bayer Pattern Unification and Bayer Preserving Augmentation

TL;DR

This paper introduces novel data pre-processing and augmentation techniques for raw image denoising that handle various Bayer patterns and preserve raw data integrity, leading to improved denoising performance.

Contribution

It proposes BayerUnify for unifying different Bayer patterns and BayerAug for raw image augmentation, enabling effective training on heterogeneous raw datasets.

Findings

Achieved state-of-the-art PSNR of 52.11 on NTIRE 2019 dataset.

Demonstrated the effectiveness of BayerUnify and BayerAug in improving raw image denoising.

Validated the approach with competitive SSIM of 0.9969.

Abstract

In this paper, we present new data pre-processing and augmentation techniques for DNN-based raw image denoising. Compared with traditional RGB image denoising, performing this task on direct camera sensor readings presents new challenges such as how to effectively handle various Bayer patterns from different data sources, and subsequently how to perform valid data augmentation with raw images. To address the first problem, we propose a Bayer pattern unification (BayerUnify) method to unify different Bayer patterns. This allows us to fully utilize a heterogeneous dataset to train a single denoising model instead of training one model for each pattern. Furthermore, while it is essential to augment the dataset to improve model generalization and performance, we discovered that it is error-prone to modify raw images by adapting augmentation methods designed for RGB images. Towards this end, we present a Bayer preserving augmentation (BayerAug) method as an effective approach for raw image augmentation. Combining these data processing technqiues with a modified U-Net, our method achieves a PSNR of 52.11 and a SSIM of 0.9969 in NTIRE 2019 Real Image Denoising Challenge, demonstrating the state-of-the-art performance. Our code is available at https://github.com/Jiaming-Liu/BayerUnifyAug.