Physically-admissible polarimetric data augmentation for road-scene analysis

TL;DR

This paper introduces a physically-constrained CycleGAN approach to generate realistic polarimetric images from labeled road scene datasets, enhancing deep learning performance in scene analysis tasks.

Contribution

It presents a novel CycleGAN-based data augmentation method that incorporates physical constraints of polarimetric imaging, improving robustness in road scene analysis.

Findings

Improved detection accuracy for cars and pedestrians by up to 9%.

Generated polarimetric images are realistic and physically feasible.

Public release of the constrained CycleGAN model.

Abstract

Polarimetric imaging, along with deep learning, has shown improved performances on different tasks including scene analysis. However, its robustness may be questioned because of the small size of the training datasets. Though the issue could be solved by data augmentation, polarization modalities are subject to physical feasibility constraints unaddressed by classical data augmentation techniques. To address this issue, we propose to use CycleGAN, an image translation technique based on deep generative models that solely relies on unpaired data, to transfer large labeled road scene datasets to the polarimetric domain. We design several auxiliary loss terms that, alongside the CycleGAN losses, deal with the physical constraints of polarimetric images. The efficiency of this solution is demonstrated on road scene object detection tasks where generated realistic polarimetric images allow to improve performances on cars and pedestrian detection up to 9%. The resulting constrained CycleGAN is publicly released, allowing anyone to generate their own polarimetric images.