SFGFusion: Surface Fitting Guided 3D Object Detection with 4D Radar and Camera Fusion

TL;DR

SFGFusion introduces a surface fitting guided fusion network for 3D object detection that effectively combines 4D radar and camera data, improving spatial representation and detection accuracy in autonomous driving.

Contribution

The paper proposes a novel surface fitting guided multi-modal fusion approach that enhances 3D detection by explicitly modeling object surfaces from radar and camera data.

Findings

Achieves superior detection performance on TJ4DRadSet and VoD benchmarks.

Effectively mitigates radar point sparsity with dense pseudo-point clouds.

Enhances spatial mapping accuracy through surface-guided feature transformation.

Abstract

3D object detection is essential for autonomous driving. As an emerging sensor, 4D imaging radar offers advantages as low cost, long-range detection, and accurate velocity measurement, making it highly suitable for object detection. However, its sparse point clouds and low resolution limit object geometric representation and hinder multi-modal fusion. In this study, we introduce SFGFusion, a novel camera-4D imaging radar detection network guided by surface fitting. By estimating quadratic surface parameters of objects from image and radar data, the explicit surface fitting model enhances spatial representation and cross-modal interaction, enabling more reliable prediction of fine-grained dense depth. The predicted depth serves two purposes: 1) in an image branch to guide the transformation of image features from perspective view (PV) to a unified bird's-eye view (BEV) for multi-modal fusion, improving spatial mapping accuracy; and 2) in a surface pseudo-point branch to generate dense pseudo-point cloud, mitigating the radar point sparsity. The original radar point cloud is also encoded in a separate radar branch. These two point cloud branches adopt a pillar-based method and subsequently transform the features into the BEV space. Finally, a standard 2D backbone and detection head are used to predict object labels and bounding boxes from BEV features. Experimental results show that SFGFusion effectively fuses camera and 4D radar features, achieving superior performance on the TJ4DRadSet and view-of-delft (VoD) object detection benchmarks.