POL-LWIR Vehicle Detection: Convolutional Neural Networks Meet Polarised Infrared Sensors

TL;DR

This paper explores vehicle detection using polarised LWIR sensors combined with CNNs, comparing different models and image decompositions to optimize accuracy and processing speed in challenging conditions.

Contribution

It introduces a novel application of polarised LWIR data with CNN configurations for vehicle detection, evaluating the trade-offs between detection accuracy and processing speed.

Findings

Faster-RCNN achieved 80.94% mAP at 6.4 fps

MobileNet SSD achieved 64.51% mAP at 53.4 fps

Polarised LWIR data enhances vehicle detection in night and adverse weather

Abstract

For vehicle autonomy, driver assistance and situational awareness, it is necessary to operate at day and night, and in all weather conditions. In particular, long wave infrared (LWIR) sensors that receive predominantly emitted radiation have the capability to operate at night as well as during the day. In this work, we employ a polarised LWIR (POL-LWIR) camera to acquire data from a mobile vehicle, to compare and contrast four different convolutional neural network (CNN) configurations to detect other vehicles in video sequences. We evaluate two distinct and promising approaches, two-stage detection (Faster-RCNN) and one-stage detection (SSD), in four different configurations. We also employ two different image decompositions: the first based on the polarisation ellipse and the second on the Stokes parameters themselves. To evaluate our approach, the experimental trials were quantified by mean average precision (mAP) and processing time, showing a clear trade-off between the two factors. For example, the best mAP result of 80.94% was achieved using Faster-RCNN, but at a frame rate of 6.4 fps. In contrast, MobileNet SSD achieved only 64.51% mAP, but at 53.4 fps.