SDVRF: Sparse-to-Dense Voxel Region Fusion for Multi-modal 3D Object Detection

TL;DR

This paper introduces SDVRF, a novel multi-modal 3D object detection method that dynamically fuses sparse LiDAR point clouds with dense image features using voxel regions, improving detection especially for small objects.

Contribution

The paper proposes a dynamic voxel region concept and a multi-scale fusion framework to enhance multi-modal 3D detection by better aligning and densifying feature fusion.

Findings

Improves detection accuracy on KITTI dataset, especially for small objects.

Outperforms baseline methods in multi-modal 3D object detection.

Enhances feature alignment and fusion density through dynamic regions.

Abstract

In the perception task of autonomous driving, multi-modal methods have become a trend due to the complementary characteristics of LiDAR point clouds and image data. However, the performance of multi-modal methods is usually limited by the sparsity of the point cloud or the noise problem caused by the misalignment between LiDAR and the camera. To solve these two problems, we present a new concept, Voxel Region (VR), which is obtained by projecting the sparse local point clouds in each voxel dynamically. And we propose a novel fusion method named Sparse-to-Dense Voxel Region Fusion (SDVRF). Specifically, more pixels of the image feature map inside the VR are gathered to supplement the voxel feature extracted from sparse points and achieve denser fusion. Meanwhile, different from prior methods, which project the size-fixed grids, our strategy of generating dynamic regions achieves better alignment and avoids introducing too much background noise. Furthermore, we propose a multi-scale fusion framework to extract more contextual information and capture the features of objects of different sizes. Experiments on the KITTI dataset show that our method improves the performance of different baselines, especially on classes of small size, including Pedestrian and Cyclist.