Does Redundancy in AI Perception Systems Help to Test for Super-Human Automated Driving Performance?

TL;DR

This paper investigates whether redundancy in AI perception systems genuinely enhances testing for super-human automated driving, revealing that correlated errors among neural networks limit the effectiveness of redundancy strategies.

Contribution

It provides experimental evidence that neural network errors are often correlated, challenging the assumption that independent subsystems improve reliability through redundancy.

Findings

Neural network errors show correlated occurrences even with different training setups.

Using multiple sensors reduces error correlation but not sufficiently to fully leverage redundancy.

Redundancy strategies may not guarantee super-human performance due to correlated errors.

Abstract

While automated driving is often advertised with better-than-human driving performance, this work reviews that it is nearly impossible to provide direct statistical evidence on the system level that this is actually the case. The amount of labeled data needed would exceed dimensions of present day technical and economical capabilities. A commonly used strategy therefore is the use of redundancy along with the proof of sufficient subsystems' performances. As it is known, this strategy is efficient especially for the case of subsystems operating independently, i.e. the occurrence of errors is independent in a statistical sense. Here, we give some first considerations and experimental evidence that this strategy is not a free ride as the errors of neural networks fulfilling the same computer vision task, at least for some cases, show correlated occurrences of errors. This remains true, if training data, architecture, and training are kept separate or independence is trained using special loss functions. Using data from different sensors (realized by up to five 2D projections of the 3D MNIST data set) in our experiments is more efficiently reducing correlations, however not to an extent that is realizing the potential of reduction of testing data that can be obtained for redundant and statistically independent subsystems.