Cross-modal semantic segmentation for indoor environmental perception using single-chip millimeter-wave radar raw data

TL;DR

This paper presents a cross-modal semantic segmentation model using single-chip millimeter-wave radar and LiDAR data for indoor environment perception, enhancing rescue operations with improved accuracy and robustness.

Contribution

It introduces a novel U-Net based segmentation model with spatial attention, and an automatic label generation method combining LiDAR and occupancy maps, tailored for mmWave radar data.

Findings

Segmentation performance is robust to azimuth variations.

Using RD tensors outperforms RA tensors for input data.

Performance declines with distance but can be mitigated.

Abstract

In the context of firefighting and rescue operations, a cross-modal semantic segmentation model based on a single-chip millimeter-wave (mmWave) radar for indoor environmental perception is proposed and discussed. To efficiently obtain high-quality labels, an automatic label generation method utilizing LiDAR point clouds and occupancy grid maps is introduced. The proposed segmentation model is based on U-Net. A spatial attention module is incorporated, which enhanced the performance of the mode. The results demonstrate that cross-modal semantic segmentation provides a more intuitive and accurate representation of indoor environments. Unlike traditional methods, the model's segmentation performance is minimally affected by azimuth. Although performance declines with increasing distance, this can be mitigated by a well-designed model. Additionally, it was found that using raw ADC data as input is ineffective; compared to RA tensors, RD tensors are more suitable for the proposed model.