Multimodal Signal Processing For Thermo-Visible-Lidar Fusion In Real-time 3D Semantic Mapping

TL;DR

This paper introduces a real-time multimodal fusion method combining visible, infrared, and LiDAR data to produce semantically enriched 3D maps with thermal information for improved autonomous navigation and environment understanding.

Contribution

It presents a novel real-time fusion technique that integrates thermal data into 3D maps, enhancing semantic understanding for complex environments.

Findings

Effective thermal feature segmentation in 3D maps

Real-time fusion of multimodal sensor data

Enhanced environment semantic mapping

Abstract

In complex environments, autonomous robot navigation and environmental perception pose higher requirements for SLAM technology. This paper presents a novel method for semantically enhancing 3D point cloud maps with thermal information. By first performing pixel-level fusion of visible and infrared images, the system projects real-time LiDAR point clouds onto this fused image stream. It then segments heat source features in the thermal channel to instantly identify high temperature targets and applies this temperature information as a semantic layer on the final 3D map. This approach generates maps that not only have accurate geometry but also possess a critical semantic understanding of the environment, making it highly valuable for specific applications like rapid disaster assessment and industrial preventive maintenance.