Driving Behavior Explanation with Multi-level Fusion

TL;DR

This paper introduces BEEF, a deep learning architecture that explains autonomous vehicle driving decisions by fusing multi-level features, validated on HDD and BDD-X datasets.

Contribution

The paper presents BEEF, a novel multi-level feature fusion model for explaining driving behavior in autonomous vehicles, trained with human decision annotations.

Findings

BEEF effectively explains driving decisions.

The model outperforms baselines in interpretability.

Validated on HDD and BDD-X datasets.

Abstract

In this era of active development of autonomous vehicles, it becomes crucial to provide driving systems with the capacity to explain their decisions. In this work, we focus on generating high-level driving explanations as the vehicle drives. We present BEEF, for BEhavior Explanation with Fusion, a deep architecture which explains the behavior of a trajectory prediction model. Supervised by annotations of human driving decisions justifications, BEEF learns to fuse features from multiple levels. Leveraging recent advances in the multi-modal fusion literature, BEEF is carefully designed to model the correlations between high-level decisions features and mid-level perceptual features. The flexibility and efficiency of our approach are validated with extensive experiments on the HDD and BDD-X datasets.