Detecting Multi-Sensor Fusion Errors in Advanced Driver-Assistance Systems

TL;DR

This paper introduces FusED, an evolutionary-based framework to detect fusion errors in multi-sensor fusion for ADAS, revealing critical safety issues in industry-grade systems and suggesting improvements.

Contribution

It presents a novel domain-specific search framework, FusED, for efficiently identifying fusion errors caused by MSF methods in ADAS.

Findings

FusED identified over 150 fusion errors in tested systems.

Fusion errors can lead to serious safety hazards like car crashes.

Causality analysis confirms MSF as the root cause of detected errors.

Abstract

Advanced Driver-Assistance Systems (ADAS) have been thriving and widely deployed in recent years. In general, these systems receive sensor data, compute driving decisions, and output control signals to the vehicles. To smooth out the uncertainties brought by sensor outputs, they usually leverage multi-sensor fusion (MSF) to fuse the sensor outputs and produce a more reliable understanding of the surroundings. However, MSF cannot completely eliminate the uncertainties since it lacks the knowledge about which sensor provides the most accurate data and how to optimally integrate the data provided by the sensors. As a result, critical consequences might happen unexpectedly. In this work, we observed that the popular MSF methods in an industry-grade ADAS can mislead the car control and result in serious safety hazards. We define the failures (e.g., car crashes) caused by the faulty MSF as fusion errors and develop a novel evolutionary-based domain-specific search framework, FusED, for the efficient detection of fusion errors. We further apply causality analysis to show that the found fusion errors are indeed caused by the MSF method. We evaluate our framework on two widely used MSF methods in two driving environments. Experimental results show that FusED identifies more than 150 fusion errors. Finally, we provide several suggestions to improve the MSF methods we study.