4D-ROLLS: 4D Radar Occupancy Learning via LiDAR Supervision

TL;DR

4D-ROLLS introduces a novel weakly supervised method for 4D radar occupancy estimation using LiDAR supervision, enhancing robustness in degraded environments and enabling fast inference for autonomous vehicle perception.

Contribution

The paper presents the first weakly supervised 4D radar occupancy estimation approach leveraging LiDAR data for supervision, with multi-stage pseudo-label generation and cross-dataset transfer capabilities.

Findings

Outperforms existing methods in degraded conditions

Achieves about 30 Hz inference speed on a 4060 GPU

Effectively transfers to downstream perception tasks

Abstract

A comprehensive understanding of 3D scenes is essential for autonomous vehicles (AVs), and among various perception tasks, occupancy estimation plays a central role by providing a general representation of drivable and occupied space. However, most existing occupancy estimation methods rely on LiDAR or cameras, which perform poorly in degraded environments such as smoke, rain, snow, and fog. In this paper, we propose 4D-ROLLS, the first weakly supervised occupancy estimation method for 4D radar using the LiDAR point cloud as the supervisory signal. Specifically, we introduce a method for generating pseudo-LiDAR labels, including occupancy queries and LiDAR height maps, as multi-stage supervision to train the 4D radar occupancy estimation model. Then the model is aligned with the occupancy map produced by LiDAR, fine-tuning its accuracy in occupancy estimation. Extensive comparative experiments validate the exceptional performance of 4D-ROLLS. Its robustness in degraded environments and effectiveness in cross-dataset training are qualitatively demonstrated. The model is also seamlessly transferred to downstream tasks BEV segmentation and point cloud occupancy prediction, highlighting its potential for broader applications. The lightweight network enables 4D-ROLLS model to achieve fast inference speeds at about 30 Hz on a 4060 GPU. The code of 4D-ROLLS will be made available at https://github.com/CLASS-Lab/4D-ROLLS.