Towards Explainable Motion Prediction using Heterogeneous Graph Representations

TL;DR

This paper enhances the interpretability of motion prediction models for autonomous vehicles by introducing a novel heterogeneous graph approach, leveraging attention mechanisms, GNNExplainer, and counterfactual reasoning to explain agent interactions.

Contribution

It proposes a new explainable heterogeneous graph-based model, XHGP, and explores multiple explanation techniques including attention, GNNExplainer, and counterfactual reasoning for motion prediction.

Findings

XHGP effectively identifies key agents and interactions.

Attention mechanisms improve interpretability of scene dynamics.

Counterfactual analysis reveals model sensitivities.

Abstract

Motion prediction systems aim to capture the future behavior of traffic scenarios enabling autonomous vehicles to perform safe and efficient planning. The evolution of these scenarios is highly uncertain and depends on the interactions of agents with static and dynamic objects in the scene. GNN-based approaches have recently gained attention as they are well suited to naturally model these interactions. However, one of the main challenges that remains unexplored is how to address the complexity and opacity of these models in order to deal with the transparency requirements for autonomous driving systems, which includes aspects such as interpretability and explainability. In this work, we aim to improve the explainability of motion prediction systems by using different approaches. First, we propose a new Explainable Heterogeneous Graph-based Policy (XHGP) model based on an heterograph representation of the traffic scene and lane-graph traversals, which learns interaction behaviors using object-level and type-level attention. This learned attention provides information about the most important agents and interactions in the scene. Second, we explore this same idea with the explanations provided by GNNExplainer. Third, we apply counterfactual reasoning to provide explanations of selected individual scenarios by exploring the sensitivity of the trained model to changes made to the input data, i.e., masking some elements of the scene, modifying trajectories, and adding or removing dynamic agents. The explainability analysis provided in this paper is a first step towards more transparent and reliable motion prediction systems, important from the perspective of the user, developers and regulatory agencies. The code to reproduce this work is publicly available at https://github.com/sancarlim/Explainable-MP/tree/v1.1.