Multi-level and multi-modal feature fusion for accurate 3D object detection in Connected and Automated Vehicles

TL;DR

This paper introduces a multi-level, multi-modal feature fusion method using a deep neural network for highly accurate 3D object detection in connected and automated vehicles, improving detection of distant and occluded objects.

Contribution

It develops a novel LIDAR-Camera fusion scheme within a three-stage feature extractor, enhancing feature representation for 3D object detection.

Findings

Outperforms recent methods on KITTI benchmarks

Improves detection accuracy for distant objects

Enhances detection of occluded instances

Abstract

Aiming at highly accurate object detection for connected and automated vehicles (CAVs), this paper presents a Deep Neural Network based 3D object detection model that leverages a three-stage feature extractor by developing a novel LIDAR-Camera fusion scheme. The proposed feature extractor extracts high-level features from two input sensory modalities and recovers the important features discarded during the convolutional process. The novel fusion scheme effectively fuses features across sensory modalities and convolutional layers to find the best representative global features. The fused features are shared by a two-stage network: the region proposal network (RPN) and the detection head (DH). The RPN generates high-recall proposals, and the DH produces final detection results. The experimental results show the proposed model outperforms more recent research on the KITTI 2D and 3D detection benchmark, particularly for distant and highly occluded instances.