Fairness in Autonomous Driving: Towards Understanding Confounding Factors in Object Detection under Challenging Weather

TL;DR

This paper investigates fairness issues in autonomous vehicle pedestrian detection, analyzing how demographic and environmental factors influence detection accuracy using novel metrics and comprehensive empirical methods.

Contribution

It introduces new probability-based fairness metrics and provides an extensive empirical analysis of bias factors affecting pedestrian detection in challenging weather conditions.

Findings

Detection performance varies with demographic attributes like gender and skin tone.

Weather severity and pedestrian proximity significantly impact detection accuracy.

Scene composition and distribution of demographic groups influence fairness in detection.

Abstract

The deployment of autonomous vehicles (AVs) is rapidly expanding to numerous cities. At the heart of AVs, the object detection module assumes a paramount role, directly influencing all downstream decision-making tasks by considering the presence of nearby pedestrians, vehicles, and more. Despite high accuracy of pedestrians detected on held-out datasets, the potential presence of algorithmic bias in such object detectors, particularly in challenging weather conditions, remains unclear. This study provides a comprehensive empirical analysis of fairness in detecting pedestrians in a state-of-the-art transformer-based object detector. In addition to classical metrics, we introduce novel probability-based metrics to measure various intricate properties of object detection. Leveraging the state-of-the-art FACET dataset and the Carla high-fidelity vehicle simulator, our analysis explores the effect of protected attributes such as gender, skin tone, and body size on object detection performance in varying environmental conditions such as ambient darkness and fog. Our quantitative analysis reveals how the previously overlooked yet intuitive factors, such as the distribution of demographic groups in the scene, the severity of weather, the pedestrians' proximity to the AV, among others, affect object detection performance. Our code is available at https://github.com/bimsarapathiraja/fair-AV.