MMDR: A Result Feature Fusion Object Detection Approach for Autonomous System

TL;DR

The paper introduces MMDR, a multimodal object detection method that fuses outcome features from different modalities at a later stage, improving detection accuracy for autonomous systems in both 2D and 3D tasks.

Contribution

It proposes a novel result feature-level fusion approach and a post-fusing network that enhances multimodal object detection by leveraging deep and shallow features.

Findings

Improved detection accuracy over previous multimodal models

Effective fusion of deep-level and global features

Applicable to both 2D and 3D object detection tasks

Abstract

Object detection has been extensively utilized in autonomous systems in recent years, encompassing both 2D and 3D object detection. Recent research in this field has primarily centered around multimodal approaches for addressing this issue.In this paper, a multimodal fusion approach based on result feature-level fusion is proposed. This method utilizes the outcome features generated from single modality sources, and fuses them for downstream tasks.Based on this method, a new post-fusing network is proposed for multimodal object detection, which leverages the single modality outcomes as features. The proposed approach, called Multi-Modal Detector based on Result features (MMDR), is designed to work for both 2D and 3D object detection tasks. Compared to previous multimodal models, the proposed approach in this paper performs feature fusion at a later stage, enabling better representation of the deep-level features of single modality sources. Additionally, the MMDR model incorporates shallow global features during the feature fusion stage, endowing the model with the ability to perceive background information and the overall input, thereby avoiding issues such as missed detections.