Simultaneous Acquisition of High Quality RGB Image and Polarization Information using a Sparse Polarization Sensor

TL;DR

This paper introduces a novel sparse polarization sensor and a dual-network system that enhances both RGB image quality and polarization information resolution simultaneously, overcoming traditional trade-offs.

Contribution

It presents a new sensor design and network architecture that jointly improves RGB image quality and polarization data resolution using sparse polarization pixels and image-guided compensation.

Findings

Outperforms state-of-the-art in polarization information compensation

Achieves higher quality RGB images with polarization data simultaneously

Provides datasets and code for further research

Abstract

This paper proposes a novel polarization sensor structure and network architecture to obtain a high-quality RGB image and polarization information. Conventional polarization sensors can simultaneously acquire RGB images and polarization information, but the polarizers on the sensor degrade the quality of the RGB images. There is a trade-off between the quality of the RGB image and polarization information as fewer polarization pixels reduce the degradation of the RGB image but decrease the resolution of polarization information. Therefore, we propose an approach that resolves the trade-off by sparsely arranging polarization pixels on the sensor and compensating for low-resolution polarization information with higher resolution using the RGB image as a guide. Our proposed network architecture consists of an RGB image refinement network and a polarization information compensation network. We confirmed the superiority of our proposed network in compensating the differential component of polarization intensity by comparing its performance with state-of-the-art methods for similar tasks: depth completion. Furthermore, we confirmed that our approach could simultaneously acquire higher quality RGB images and polarization information than conventional polarization sensors, resolving the trade-off between the quality of RGB images and polarization information. The baseline code and newly generated real and synthetic large-scale polarization image datasets are available for further research and development.