Methodology for an Analysis of Influencing Factors on 3D Object Detection Performance

TL;DR

This paper introduces a new methodology combining statistical analysis and machine learning to understand how various factors influence the performance of 3D object detectors in automated driving, aiming to improve safety.

Contribution

It presents a novel approach that integrates univariate analysis and Random Forest models with Shapley Values to analyze influencing factors on 3D object detection performance.

Findings

Identifies key environmental and object-related factors affecting detection errors

Uses RF and Shapley Values for interpretable error prediction

Reveals performance gaps in current 3D detectors

Abstract

In automated driving, object detection is crucial for perceiving the environment. Although deep learning-based detectors offer high performance, their black-box nature complicates safety assurance. We propose a novel methodology to analyze how object- and environment-related factors affect LiDAR- and camera-based 3D object detectors. A statistical univariate analysis relates each factor to pedestrian detection errors. Additionally, a Random Forest (RF) model predicts errors from meta-information, with Shapley Values interpreting feature importance. By capturing feature dependencies, the RF enables a nuanced analysis of detection errors. Understanding these factors reveals detector performance gaps and supports safer object detection system development.