IRisPath: Enhancing Costmap for Off-Road Navigation with Robust IR-RGB Fusion for Improved Day and Night Traversability

TL;DR

IRisPath is a multi-modal fusion network that combines Thermal and RGB images to improve off-road navigation robustness across day and night conditions, supported by a new dataset and calibration method.

Contribution

The paper introduces IRisPath, a novel fusion network for off-road navigation that handles dynamic lighting and weather, along with a new dataset and calibration technique.

Findings

IRisPath improves traversability in varied lighting conditions.

The dataset enables better training for off-road perception models.

The calibration method achieves high accuracy in sensor alignment.

Abstract

Autonomous off-road navigation is required for applications in agriculture, construction, search and rescue and defence. Traditional on-road autonomous methods struggle with dynamic terrains, leading to poor vehicle control in off-road conditions. Recent deep-learning models have used perception sensors along with kinesthetic feedback for navigation on such terrains. However, this approach has out-of-domain uncertainty. Factors like change in time of day and weather impacts the performance of the model. We propose a multi modal fusion network "IRisPath" capable of using Thermal and RGB images to provide robustness against dynamic weather and light conditions. To aid further works in this domain, we also open-source a day-night dataset with Thermal and RGB images along with pseudo-labels for traversability. In order to co-register for fusion model we also develop a novel method for targetless extrinsic calibration of Thermal, LiDAR and RGB cameras with translation accuracy of +/-1.7cm and rotation accuracy of +/-0.827degrees.