Deep convolutional demosaicking network for multispectral polarization filter array

TL;DR

This paper introduces MSPDNet, a deep learning model that effectively reconstructs high-quality multispectral polarization images from mosaic data, advancing imaging accuracy in MSPFA systems.

Contribution

The paper presents a novel deep convolutional network that leverages 3D convolution and deep learning for improved multispectral polarization demosaicking.

Findings

MSPDNet achieves higher PSNR than existing methods.

The method accurately reconstructs multi-wavelength and polarization information.

Visual quality of reconstructed images is significantly improved.

Abstract

To address the demosaicking problem in multispectral polarization filter array (MSPFA) imaging, we propose a multispectral polarization demosaicking network (MSPDNet) that improves image reconstruction accuracy. Imaging with a multispectral polarization filter array acquires multispectral polarization information in a snapshot. The full-resolution multispectral polarization image must be reconstructed from a mosaic image. In the proposed method, a sparse image in which pixel values of the same channel are extracted from a mosaic image is used as input to MSPDNet. Missing pixels are interpolated by learning spatial and wavelength correlations from the observed pixels in the mosaic image. Moreover, by using 3D convolution, features are extracted at each convolution layer, and by deepening the network, even detailed features of the multispectral polarization image can be learned. Experimental results show that MSPDNet can reconstruct multi-wavelength and multi-polarization angle information with high accuracy in terms of peak signal-to-noise ratio (PSNR) evaluation and visual quality, indicating the effectiveness of the proposed method compared to other methods.