Inheriting Bayer's Legacy-Joint Remosaicing and Denoising for Quad Bayer Image Sensor

TL;DR

This paper introduces DJRD, a dual-head network that jointly remosaics and denoises Quad Bayer images, improving low-light imaging resolution and color accuracy without extra hardware complexity.

Contribution

The paper presents a novel joint remosaicing and denoising network with a specialized QB-Re block and advanced modules, achieving superior performance in low-light conditions.

Findings

Outperforms competing models by approximately 3dB in noise reduction.

Effectively reduces color misalignment and artifacts in Quad Bayer images.

Enhances low-light image quality without additional hardware complexity.

Abstract

Pixel binning based Quad sensors have emerged as a promising solution to overcome the hardware limitations of compact cameras in low-light imaging. However, binning results in lower spatial resolution and non-Bayer CFA artifacts. To address these challenges, we propose a dual-head joint remosaicing and denoising network (DJRD), which enables the conversion of noisy Quad Bayer and standard noise-free Bayer pattern without any resolution loss. DJRD includes a newly designed Quad Bayer remosaicing (QB-Re) block, integrated denoising modules based on Swin-transformer and multi-scale wavelet transform. The QB-Re block constructs the convolution kernel based on the CFA pattern to achieve a periodic color distribution in the perceptual field, which is used to extract exact spectral information and reduce color misalignment. The integrated Swin-Transformer and multi-scale wavelet transform capture non-local dependencies, frequency and location information to effectively reduce practical noise. By identifying challenging patches utilizing Moire and zipper detection metrics, we enable our model to concentrate on difficult patches during the post-training phase, which enhances the model's performance in hard cases. Our proposed model outperforms competing models by approximately 3dB, without additional complexity in hardware or software.