LoGoNet: Towards Accurate 3D Object Detection with Local-to-Global Cross-Modal Fusion

TL;DR

LoGoNet introduces a novel local-to-global fusion approach for LiDAR-camera data, significantly improving 3D object detection accuracy by combining fine-grained region-level and scene-level features.

Contribution

The paper proposes LoGoNet, a fusion network that integrates local and global features using point centroids and grid-based image sampling, advancing multi-modal 3D detection.

Findings

Achieves state-of-the-art results on Waymo and KITTI datasets.

Ranks 1st on Waymo 3D detection leaderboard.

Surpasses 80 APH (L2) on three classes simultaneously.

Abstract

LiDAR-camera fusion methods have shown impressive performance in 3D object detection. Recent advanced multi-modal methods mainly perform global fusion, where image features and point cloud features are fused across the whole scene. Such practice lacks fine-grained region-level information, yielding suboptimal fusion performance. In this paper, we present the novel Local-to-Global fusion network (LoGoNet), which performs LiDAR-camera fusion at both local and global levels. Concretely, the Global Fusion (GoF) of LoGoNet is built upon previous literature, while we exclusively use point centroids to more precisely represent the position of voxel features, thus achieving better cross-modal alignment. As to the Local Fusion (LoF), we first divide each proposal into uniform grids and then project these grid centers to the images. The image features around the projected grid points are sampled to be fused with position-decorated point cloud features, maximally utilizing the rich contextual information around the proposals. The Feature Dynamic Aggregation (FDA) module is further proposed to achieve information interaction between these locally and globally fused features, thus producing more informative multi-modal features. Extensive experiments on both Waymo Open Dataset (WOD) and KITTI datasets show that LoGoNet outperforms all state-of-the-art 3D detection methods. Notably, LoGoNet ranks 1st on Waymo 3D object detection leaderboard and obtains 81.02 mAPH (L2) detection performance. It is noteworthy that, for the first time, the detection performance on three classes surpasses 80 APH (L2) simultaneously. Code will be available at \url{https://github.com/sankin97/LoGoNet}.