Towards Sensor Data Abstraction of Autonomous Vehicle Perception Systems

TL;DR

This paper reviews sensor modalities and explores data abstraction techniques to improve the transferability of perception models across different sensor setups in autonomous vehicles.

Contribution

It provides a comprehensive review of sensor modalities and identifies key paths for abstracting sensor data to enhance model transferability.

Findings

Analysis of camera, lidar, and radar modalities in datasets

Examination of single sensor and multi-sensor abstraction methods

Identification of critical steps towards sensor data abstraction

Abstract

Full-stack autonomous driving perception modules usually consist of data-driven models based on multiple sensor modalities. However, these models might be biased to the sensor setup used for data acquisition. This bias can seriously impair the perception models' transferability to new sensor setups, which continuously occur due to the market's competitive nature. We envision sensor data abstraction as an interface between sensor data and machine learning applications for highly automated vehicles (HAD). For this purpose, we review the primary sensor modalities, camera, lidar, and radar, published in autonomous-driving related datasets, examine single sensor abstraction and abstraction of sensor setups, and identify critical paths towards an abstraction of sensor data from multiple perception configurations.