LEROjD: Lidar Extended Radar-Only Object Detection

TL;DR

This paper introduces methods to improve radar-only 3D object detection for autonomous driving by transferring knowledge from lidar data through multi-stage training and cross-modal distillation, achieving significant accuracy gains.

Contribution

It proposes two novel strategies for training radar-only detectors using lidar data without changing detector architectures, enhancing detection performance.

Findings

Up to 4.2% improvement in mean Average Precision

Effective knowledge transfer via multi-stage training and distillation

Applicable to multiple 3D detection networks

Abstract

Accurate 3D object detection is vital for automated driving. While lidar sensors are well suited for this task, they are expensive and have limitations in adverse weather conditions. 3+1D imaging radar sensors offer a cost-effective, robust alternative but face challenges due to their low resolution and high measurement noise. Existing 3+1D imaging radar datasets include radar and lidar data, enabling cross-modal model improvements. Although lidar should not be used during inference, it can aid the training of radar-only object detectors. We explore two strategies to transfer knowledge from the lidar to the radar domain and radar-only object detectors: 1. multi-stage training with sequential lidar point cloud thin-out, and 2. cross-modal knowledge distillation. In the multi-stage process, three thin-out methods are examined. Our results show significant performance gains of up to 4.2 percentage points in mean Average Precision with multi-stage training and up to 3.9 percentage points with knowledge distillation by initializing the student with the teacher's weights. The main benefit of these approaches is their applicability to other 3D object detection networks without altering their architecture, as we show by analyzing it on two different object detectors. Our code is available at https://github.com/rst-tu-dortmund/lerojd