Efficient Deep Demosaicing with Spatially Downsampled Isotropic Networks

TL;DR

This paper proposes that incorporating spatial downsampling into isotropic networks enhances both efficiency and performance in deep learning-based image demosaicing, especially suitable for mobile applications.

Contribution

It introduces a novel approach of using spatial downsampling in isotropic networks for demosaicing, demonstrating improved efficiency and accuracy over traditional designs.

Findings

Downsampling improves empirical performance of isotropic networks.

JD3Net achieves strong results on demosaicing and joint denoising tasks.

Spatial downsampling reduces computational cost for mobile image processing.

Abstract

In digital imaging, image demosaicing is a crucial first step which recovers the RGB information from a color filter array (CFA). Oftentimes, deep learning is utilized to perform image demosaicing. Given that most modern digital imaging applications occur on mobile platforms, applying deep learning to demosaicing requires lightweight and efficient networks. Isotropic networks, also known as residual-in-residual networks, have been often employed for image demosaicing and joint-demosaicing-and-denoising (JDD). Most demosaicing isotropic networks avoid spatial downsampling entirely, and thus are often prohibitively expensive computationally for mobile applications. Contrary to previous isotropic network designs, this paper claims that spatial downsampling to a signficant degree can improve the efficiency and performance of isotropic networks. To validate this claim, we design simple fully convolutional networks with and without downsampling using a mathematical architecture design technique adapted from DeepMAD, and find that downsampling improves empirical performance. Additionally, empirical testing of the downsampled variant, JD3Net, of our fully convolutional networks reveals strong empirical performance on a variety of image demosaicing and JDD tasks.