Long-Range Thermal 3D Perception in Low Contrast Environments

TL;DR

This paper demonstrates a significant sensitivity enhancement in LWIR thermal sensors, enabling robust 3D perception in low contrast, degraded visual, and GPS-denied environments for autonomous aerial vehicles.

Contribution

It introduces a novel multi-sensor thermal system with increased sensitivity and improved contrast, suitable for 3D perception in challenging conditions, and details prototype design and calibration.

Findings

3.84x contrast increase in intrascene data

Additional 5.5x contrast with interscene accumulation

Achieved NETD of 1.9 mK with 40 mK sensors

Abstract

This report discusses the results of SBIR Phase I effort to prove the feasibility of dramatic improvement of the microbolometer-based Long Wave Infrared (LWIR) detectors sensitivity, especially for the 3D measurements. The resulting low SWaP-C thermal depth-sensing system will enable the situational awareness of Autonomous Air Vehicles for Advanced Air Mobility (AAM). It will provide robust 3D information of the surrounding environment, including low-contrast static and moving objects, at far distances in degraded visual conditions and GPS-denied areas. Our multi-sensor 3D perception enabled by COTS uncooled thermal sensors mitigates major weakness of LWIR sensors - low contrast by increasing the system sensitivity over an order of magnitude. There were no available thermal image sets suitable for evaluating this technology, making datasets acquisition our first goal. We discuss the design and construction of the prototype system with sixteen 640pix x 512pix LWIR detectors, camera calibration to subpixel resolution, capture, and process synchronized image. The results show the 3.84x contrast increase for intrascene-only data and an additional 5.5x - with the interscene accumulation, reaching system noise-equivalent temperature difference (NETD) of 1.9 mK with the 40 mK sensors.