DGFusion: Dual-guided Fusion for Robust Multi-Modal 3D Object Detection

TL;DR

DGFusion introduces a dual-guided fusion approach for multi-modal 3D object detection, significantly improving detection accuracy of hard instances like distant or occluded objects in autonomous driving.

Contribution

The paper proposes DGFusion, a novel dual-guided paradigm that combines Point-guide-Image and Image-guide-Point methods for enhanced multi-modal feature fusion in 3D detection.

Findings

Outperforms baseline methods with +1.0% mAP and +0.8% NDS improvements.

Demonstrates robust detection of hard instances across various scenarios.

Achieves consistent gains in small-scale training and occlusion conditions.

Abstract

As a critical task in autonomous driving perception systems, 3D object detection is used to identify and track key objects, such as vehicles and pedestrians. However, detecting distant, small, or occluded objects (hard instances) remains a challenge, which directly compromises the safety of autonomous driving systems. We observe that existing multi-modal 3D object detection methods often follow a single-guided paradigm, failing to account for the differences in information density of hard instances between modalities. In this work, we propose DGFusion, based on the Dual-guided paradigm, which fully inherits the advantages of the Point-guide-Image paradigm and integrates the Image-guide-Point paradigm to address the limitations of the single paradigms. The core of DGFusion, the Difficulty-aware Instance Pair Matcher (DIPM), performs instance-level feature matching based on difficulty to generate easy and hard instance pairs, while the Dual-guided Modules exploit the advantages of both pair types to enable effective multi-modal feature fusion. Experimental results demonstrate that our DGFusion outperforms the baseline methods, with respective improvements of +1.0\% mAP, +0.8\% NDS, and +1.3\% average recall on nuScenes. Extensive experiments demonstrate consistent robustness gains for hard instance detection across ego-distance, size, visibility, and small-scale training scenarios.