RadarNeXt: Real-Time and Reliable 3D Object Detector Based On 4D mmWave Imaging Radar

TL;DR

RadarNeXt introduces a real-time, reliable 3D object detection method using 4D mmWave radar data, combining re-parameterizable neural networks and a novel foreground enhancement network to achieve high accuracy and speed for autonomous driving.

Contribution

It proposes RadarNeXt, a novel 3D detection framework that leverages re-parameterizable neural networks and a Multi-path Deformable Foreground Enhancement Network for improved radar-based perception.

Findings

Achieves over 67 FPS on RTX A4000 GPU.

Attains 50.48 and 32.30 mAP on View-of-Delft and TJ4DRadSet datasets.

Demonstrates high accuracy and real-time performance in 3D radar object detection.

Abstract

3D object detection is crucial for Autonomous Driving (AD) and Advanced Driver Assistance Systems (ADAS). However, most 3D detectors prioritize detection accuracy, often overlooking network inference speed in practical applications. In this paper, we propose RadarNeXt, a real-time and reliable 3D object detector based on the 4D mmWave radar point clouds. It leverages the re-parameterizable neural networks to catch multi-scale features, reduce memory cost and accelerate the inference. Moreover, to highlight the irregular foreground features of radar point clouds and suppress background clutter, we propose a Multi-path Deformable Foreground Enhancement Network (MDFEN), ensuring detection accuracy while minimizing the sacrifice of speed and excessive number of parameters. Experimental results on View-of-Delft and TJ4DRadSet datasets validate the exceptional performance and efficiency of RadarNeXt, achieving 50.48 and 32.30 mAPs with the variant using our proposed MDFEN. Notably, our RadarNeXt variants achieve inference speeds of over 67.10 FPS on the RTX A4000 GPU and 28.40 FPS on the Jetson AGX Orin. This research demonstrates that RadarNeXt brings a novel and effective paradigm for 3D perception based on 4D mmWave radar.