Exploring the Causality of End-to-End Autonomous Driving

TL;DR

This paper presents a systematic approach to analyze and debug the causality in end-to-end autonomous driving models, transforming the black-box into a more interpretable system using qualitative and quantitative methods within a simulator.

Contribution

It introduces a comprehensive causality analysis framework for end-to-end autonomous driving, including validation, visualization, and a new debugging tool within the CARLA simulator.

Findings

Validated key input dependencies via counterfactuals

Developed a causality exploration tool for CARLA

Demonstrated effective debugging of autonomous driving models

Abstract

Deep learning-based models are widely deployed in autonomous driving areas, especially the increasingly noticed end-to-end solutions. However, the black-box property of these models raises concerns about their trustworthiness and safety for autonomous driving, and how to debug the causality has become a pressing concern. Despite some existing research on the explainability of autonomous driving, there is currently no systematic solution to help researchers debug and identify the key factors that lead to the final predicted action of end-to-end autonomous driving. In this work, we propose a comprehensive approach to explore and analyze the causality of end-to-end autonomous driving. First, we validate the essential information that the final planning depends on by using controlled variables and counterfactual interventions for qualitative analysis. Then, we quantitatively assess the factors influencing model decisions by visualizing and statistically analyzing the response of key model inputs. Finally, based on the comprehensive study of the multi-factorial end-to-end autonomous driving system, we have developed a strong baseline and a tool for exploring causality in the close-loop simulator CARLA. It leverages the essential input sources to obtain a well-designed model, resulting in highly competitive capabilities. As far as we know, our work is the first to unveil the mystery of end-to-end autonomous driving and turn the black box into a white one. Thorough close-loop experiments demonstrate that our method can be applied to end-to-end autonomous driving solutions for causality debugging. Code will be available at https://github.com/bdvisl/DriveInsight.