SemRaFiner: Panoptic Segmentation in Sparse and Noisy Radar Point Clouds

TL;DR

SemRaFiner advances radar-based scene understanding by improving panoptic segmentation in sparse, noisy radar point clouds, addressing sensor limitations and enhancing accuracy for autonomous driving.

Contribution

Introduces SemRaFiner, a novel method that handles varying density and noise in radar data, with optimized feature extraction and training for better segmentation.

Findings

Outperforms existing radar panoptic segmentation methods

Effectively handles sparse and noisy radar data

Improves accuracy in dynamic scene understanding

Abstract

Semantic scene understanding, including the perception and classification of moving agents, is essential to enabling safe and robust driving behaviours of autonomous vehicles. Cameras and LiDARs are commonly used for semantic scene understanding. However, both sensor modalities face limitations in adverse weather and usually do not provide motion information. Radar sensors overcome these limitations and directly offer information about moving agents by measuring the Doppler velocity, but the measurements are comparably sparse and noisy. In this paper, we address the problem of panoptic segmentation in sparse radar point clouds to enhance scene understanding. Our approach, called SemRaFiner, accounts for changing density in sparse radar point clouds and optimizes the feature extraction to improve accuracy. Furthermore, we propose an optimized training procedure to refine instance assignments by incorporating a dedicated data augmentation. Our experiments suggest that our approach outperforms state-of-the-art methods for radar-based panoptic segmentation.