Polarimetric Imaging for Perception

TL;DR

This paper explores the use of RGB-polarimetric cameras in autonomous driving, demonstrating that polarization information can improve perception tasks like depth estimation and free space detection.

Contribution

It introduces a new dataset and shows that polarization data enhances perception accuracy with minimal changes to existing neural network architectures.

Findings

Polarimetric data improves depth estimation accuracy.

Polarimetric data enhances free space detection performance.

A new RGB-polarimetric dataset supports further research.

Abstract

Autonomous driving and advanced driver-assistance systems rely on a set of sensors and algorithms to perform the appropriate actions and provide alerts as a function of the driving scene. Typically, the sensors include color cameras, radar, lidar and ultrasonic sensors. Strikingly however, although light polarization is a fundamental property of light, it is seldom harnessed for perception tasks. In this work we analyze the potential for improvement in perception tasks when using an RGB-polarimetric camera, as compared to an RGB camera. We examine monocular depth estimation and free space detection during the middle of the day, when polarization is independent of subject heading, and show that a quantifiable improvement can be achieved for both of them using state-of-the-art deep neural networks, with a minimum of architectural changes. We also present a new dataset composed of RGB-polarimetric images, lidar scans, GNSS / IMU readings and free space segmentations that further supports developing perception algorithms that take advantage of light polarization.