Sensitivity analysis of AI-based algorithms for autonomous driving on optical wavefront aberrations induced by the windshield

TL;DR

This paper investigates how optical wavefront aberrations caused by windshields affect the performance of AI perception models in autonomous driving, revealing a performance gap and limitations of current optical metrics.

Contribution

It introduces a Fourier optics-based threat model to evaluate the sensitivity of perception models to windshield-induced optical aberrations, highlighting the need for improved optical metrics.

Findings

Windshields induce a measurable performance gap in perception models.

Existing optical metrics may not fully capture the impact of wavefront aberrations.

Sensitivity varies across different perception models and windshield configurations.

Abstract

Autonomous driving perception techniques are typically based on supervised machine learning models that are trained on real-world street data. A typical training process involves capturing images with a single car model and windshield configuration. However, deploying these trained models on different car types can lead to a domain shift, which can potentially hurt the neural networks performance and violate working ADAS requirements. To address this issue, this paper investigates the domain shift problem further by evaluating the sensitivity of two perception models to different windshield configurations. This is done by evaluating the dependencies between neural network benchmark metrics and optical merit functions by applying a Fourier optics based threat model. Our results show that there is a performance gap introduced by windshields and existing optical metrics used for posing requirements might not be sufficient.