An End-to-End Decision-Aware Multi-Scale Attention-Based Model for Explainable Autonomous Driving

TL;DR

This paper introduces a multi-scale attention-based model for autonomous driving that provides case-specific explanations of decisions, improving interpretability and reliability in AI systems for autonomous vehicles.

Contribution

The study presents a novel decision-aware, multi-scale attention model with a new evaluation metric, enhancing explainability and robustness in autonomous driving AI.

Findings

The proposed model outperforms state-of-the-art models in accuracy.

It introduces the Joint F1 score for better evaluation of explainability.

The model demonstrates strong generalization on multiple datasets.

Abstract

The application of computer vision is gradually increasing across various domains. They employ deep learning models with a black-box nature. Without the ability to explain the behavior of neural networks, especially their decision-making processes, it is not possible to recognize their efficiency, predict system failures, or effectively implement them in real-world applications. Due to the inevitable use of deep learning in fully automated driving systems, many methods have been proposed to explain their behavior; however, they suffer from flawed reasoning and unreliable metrics, which have prevented a comprehensive understanding of complex models in autonomous vehicles and hindered the development of truly reliable systems. In this study, we propose a multi-scale attention-based model in which driving decisions are fed into the reasoning component to provide case-specific explanations for each decision simultaneously. For quantitative evaluation of our model's performance, we employ the F1-score metric, and also proposed a new metric called the Joint F1 score to demonstrate the accurate and reliable performance of the model in terms of Explainable Artificial Intelligence (XAI). In addition to the BDD-OIA dataset, the nu-AR dataset is utilized to further validate the generalization capability and robustness of the proposed network. The results demonstrate the superiority of our reasoning network over the classic and state-of-the-art models.