Joint demosaicing and fusion of multiresolution coded acquisitions: A unified image formation and reconstruction method

TL;DR

This paper introduces a unified framework for modeling multiresolution coded acquisitions in optical imaging and proposes a joint demosaicing and fusion algorithm to reconstruct high-resolution images from various acquisition modalities.

Contribution

It presents a novel unified modeling framework for multiresolution coded acquisitions and a model-based reconstruction algorithm that jointly demosaics and fuses data for high-resolution image recovery.

Findings

Reconstruction of uncompressed image datacubes at high resolution.

Unified framework encompasses various optical imaging systems.

Effective joint demosaicing and fusion demonstrated.

Abstract

Novel optical imaging devices allow for hybrid acquisition modalities such as compressed acquisitions with locally different spatial and spectral resolutions captured by a single focal plane array. In this work, we propose to model the capturing system of a multiresolution coded acquisition (MRCA) in a unified framework, which natively includes conventional systems such as those based on spectral/color filter arrays, compressed coded apertures, and multiresolution sensing. We also propose a model-based image reconstruction algorithm performing a joint demosaicing and fusion (JoDeFu) of any acquisition modeled in the MRCA framework. The JoDeFu reconstruction algorithm solves an inverse problem with a proximal splitting technique and is able to reconstruct an uncompressed image datacube at the highest available spatial and spectral resolution. An implementation of the code is available at https://github.com/danaroth83/jodefu.