GraphFusion3D: Dynamic Graph Attention Convolution with Adaptive Cross-Modal Transformer for 3D Object Detection

TL;DR

GraphFusion3D introduces a novel multi-modal framework with adaptive transformers and graph reasoning to enhance 3D object detection from point clouds, addressing challenges of data sparsity and incomplete structures.

Contribution

The paper proposes GraphFusion3D, integrating adaptive cross-modal transformers and graph reasoning modules for improved 3D detection accuracy.

Findings

Achieves 70.6% AP25 on SUN RGB-D

Attains 75.1% AP25 on ScanNetV2

Significantly outperforms existing methods

Abstract

Despite significant progress in 3D object detection, point clouds remain challenging due to sparse data, incomplete structures, and limited semantic information. Capturing contextual relationships between distant objects presents additional difficulties. To address these challenges, we propose GraphFusion3D, a unified framework combining multi-modal fusion with advanced feature learning. Our approach introduces the Adaptive Cross-Modal Transformer (ACMT), which adaptively integrates image features into point representations to enrich both geometric and semantic information. For proposal refinement, we introduce the Graph Reasoning Module (GRM), a novel mechanism that models neighborhood relationships to simultaneously capture local geometric structures and global semantic context. The module employs multi-scale graph attention to dynamically weight both spatial proximity and feature similarity between proposals. We further employ a cascade decoder that progressively refines detections through multi-stage predictions. Extensive experiments on SUN RGB-D (70.6% AP$_{25}$ and 51.2% AP$_{50}$) and ScanNetV2 (75.1% AP$_{25}$ and 60.8% AP$_{50}$) demonstrate a substantial performance improvement over existing approaches.