A Review of an Old Dilemma: Demosaicking First, or Denoising First?

TL;DR

This paper reviews the strategies for combining denoising and demosaicking in digital camera pipelines, concluding that demosaicking should precede denoising, with adaptations needed for classic algorithms to handle demosaicked noise.

Contribution

It provides an extensive evaluation of denoising and demosaicking order strategies and proposes adaptations for denoising algorithms to improve image reconstruction.

Findings

Demosaicking first yields better full color image reconstruction.

Classic denoising algorithms need adaptation for demosaicked noise.

The study offers guidelines for implementing efficient joint processing in low-power devices.

Abstract

Image denoising and demosaicking are the most important early stages in digital camera pipelines. They constitute a severely ill-posed problem that aims at reconstructing a full color image from a noisy color filter array (CFA) image. In most of the literature, denoising and demosaicking are treated as two independent problems, without considering their interaction, or asking which should be applied first. Several recent works have started addressing them jointly in works that involve heavy weight CNNs, thus incompatible with low power portable imaging devices. Hence, the question of how to combine denoising and demosaicking to reconstruct full color images remains very relevant: Is denoising to be applied first, or should that be demosaicking first? In this paper, we review the main variants of these strategies and carry-out an extensive evaluation to find the best way to reconstruct full color images from a noisy mosaic. We conclude that demosaicking should applied first, followed by denoising. Yet we prove that this requires an adaptation of classic denoising algorithms to demosaicked noise, which we justify and specify.