Unveiling the Black Box: Independent Functional Module Evaluation for Bird's-Eye-View Perception Model

TL;DR

This paper introduces BEV-IFME, a framework for evaluating the internal modules of autonomous driving perception models by comparing feature maps to ground truth, improving interpretability and trust.

Contribution

It proposes a novel evaluation framework with a two-stage Alignment AutoEncoder to assess module training maturity through similarity scores.

Findings

Similarity Score correlates strongly with BEV metrics (average 0.9387)

Framework effectively assesses module training maturity

Enhances interpretability of end-to-end perception models

Abstract

End-to-end models are emerging as the mainstream in autonomous driving perception. However, the inability to meticulously deconstruct their internal mechanisms results in diminished development efficacy and impedes the establishment of trust. Pioneering in the issue, we present the Independent Functional Module Evaluation for Bird's-Eye-View Perception Model (BEV-IFME), a novel framework that juxtaposes the module's feature maps against Ground Truth within a unified semantic Representation Space to quantify their similarity, thereby assessing the training maturity of individual functional modules. The core of the framework lies in the process of feature map encoding and representation aligning, facilitated by our proposed two-stage Alignment AutoEncoder, which ensures the preservation of salient information and the consistency of feature structure. The metric for evaluating the training maturity of functional modules, Similarity Score, demonstrates a robust positive correlation with BEV metrics, with an average correlation coefficient of 0.9387, attesting to the framework's reliability for assessment purposes.