Explainable AI for Safe and Trustworthy Autonomous Driving: A Systematic Review

TL;DR

This paper systematically reviews explainable AI methods for autonomous driving, emphasizing their role in enhancing safety and trustworthiness by analyzing requirements, proposing a taxonomy, and introducing a modular framework called SafeX.

Contribution

It provides the first comprehensive review of XAI techniques for autonomous driving and introduces the SafeX framework for integrating explanations with safety assurance.

Findings

XAI is essential for meeting safety and trust requirements in AD

Five key contributions of XAI identified: interpretable design, surrogate models, monitoring, explanations, validation

Proposed SafeX framework enables explanation delivery and safety assurance

Abstract

Artificial Intelligence (AI) shows promising applications for the perception and planning tasks in autonomous driving (AD) due to its superior performance compared to conventional methods. However, inscrutable AI systems exacerbate the existing challenge of safety assurance of AD. One way to mitigate this challenge is to utilize explainable AI (XAI) techniques. To this end, we present the first comprehensive systematic literature review of explainable methods for safe and trustworthy AD. We begin by analyzing the requirements for AI in the context of AD, focusing on three key aspects: data, model, and agency. We find that XAI is fundamental to meeting these requirements. Based on this, we explain the sources of explanations in AI and describe a taxonomy of XAI. We then identify five key contributions of XAI for safe and trustworthy AI in AD, which are interpretable design, interpretable surrogate models, interpretable monitoring, auxiliary explanations, and interpretable validation. Finally, we propose a modular framework called SafeX to integrate these contributions, enabling explanation delivery to users while simultaneously ensuring the safety of AI models.