A survey on deep learning approaches for data integration in autonomous driving system

TL;DR

This survey reviews recent deep learning methods for multi-sensor data integration in autonomous vehicle perception, proposing a new taxonomy and analyzing their advantages and limitations.

Contribution

It introduces a novel taxonomy for data integration in autonomous driving perception based on multi-view, multi-modality, and multi-frame dimensions.

Findings

Summarizes various deep learning integration techniques and their trade-offs.

Provides insights into properties of an ideal data integration approach.

Discusses key features for optimal data integration in autonomous systems.

Abstract

The perception module of self-driving vehicles relies on a multi-sensor system to understand its environment. Recent advancements in deep learning have led to the rapid development of approaches that integrate multi-sensory measurements to enhance perception capabilities. This paper surveys the latest deep learning integration techniques applied to the perception module in autonomous driving systems, categorizing integration approaches based on "what, how, and when to integrate". A new taxonomy of integration is proposed, based on three dimensions: multi-view, multi-modality, and multi-frame. The integration operations and their pros and cons are summarized, providing new insights into the properties of an "ideal" data integration approach that can alleviate the limitations of existing methods. After reviewing hundreds of relevant papers, this survey concludes with a discussion of the key features of an optimal data integration approach.