Object Detection in Thermal Spectrum for Advanced Driver-Assistance Systems (ADAS)

TL;DR

This paper explores adapting state-of-the-art object detection frameworks to thermal infrared data for ADAS, validating on public and novel datasets, and optimizing for real-time edge deployment.

Contribution

It introduces a thermal vision object detection approach with validation on diverse datasets and demonstrates optimized deployment on Nvidia Jetson Nano.

Findings

High accuracy in low-light and challenging weather conditions

Effective model optimization with TensorRT for real-time inference

Successful deployment on resource-constrained edge hardware

Abstract

Object detection in thermal infrared spectrum provides more reliable data source in low-lighting conditions and different weather conditions, as it is useful both in-cabin and outside for pedestrian, animal, and vehicular detection as well as for detecting street-signs & lighting poles. This paper is about exploring and adapting state-of-the-art object detection and classifier framework on thermal vision with seven distinct classes for advanced driver-assistance systems (ADAS). The trained network variants on public datasets are validated on test data with three different test approaches which include test-time with no augmentation, test-time augmentation, and test-time with model ensembling. Additionally, the efficacy of trained networks is tested on locally gathered novel test-data captured with an uncooled LWIR prototype thermal camera in challenging weather and environmental scenarios. The performance analysis of trained models is investigated by computing precision, recall, and mean average precision scores (mAP). Furthermore, the trained model architecture is optimized using TensorRT inference accelerator and deployed on resource-constrained edge hardware Nvidia Jetson Nano to explicitly reduce the inference time on GPU as well as edge devices for further real-time onboard installations.