Multi-Task Cross-Modality Attention-Fusion for 2D Object Detection

TL;DR

This paper introduces a novel multi-task attention-fusion network for radar-camera data that improves 2D object detection, especially in adverse weather and nighttime conditions, by better aligning data and jointly detecting objects and free space.

Contribution

It presents two new radar preprocessing techniques and a multi-task fusion network with novel fusion blocks, advancing radar-camera fusion for autonomous driving.

Findings

Outperforms state-of-the-art in nuScenes dataset

More robust in adverse weather and night conditions

Joint detection and segmentation improves focus on relevant scene parts

Abstract

Accurate and robust object detection is critical for autonomous driving. Image-based detectors face difficulties caused by low visibility in adverse weather conditions. Thus, radar-camera fusion is of particular interest but presents challenges in optimally fusing heterogeneous data sources. To approach this issue, we propose two new radar preprocessing techniques to better align radar and camera data. In addition, we introduce a Multi-Task Cross-Modality Attention-Fusion Network (MCAF-Net) for object detection, which includes two new fusion blocks. These allow for exploiting information from the feature maps more comprehensively. The proposed algorithm jointly detects objects and segments free space, which guides the model to focus on the more relevant part of the scene, namely, the occupied space. Our approach outperforms current state-of-the-art radar-camera fusion-based object detectors in the nuScenes dataset and achieves more robust results in adverse weather conditions and nighttime scenarios.