Noise-Free Explanation for Driving Action Prediction

TL;DR

This paper introduces SNNA, a novel attention explanation method that produces noise-free, clear visual explanations for complex multi-label driving action prediction models, outperforming existing methods.

Contribution

The paper proposes SNNA, an effective attention weighting technique that reduces noise in explanations, specifically tailored for multi-label driving action prediction tasks.

Findings

SNNA generates clearer visual explanation maps.

SNNA outperforms state-of-the-art methods in pixel importance ranking.

SNNA improves trustworthiness of model explanations.

Abstract

Although attention mechanisms have achieved considerable progress in Transformer-based architectures across various Artificial Intelligence (AI) domains, their inner workings remain to be explored. Existing explainable methods have different emphases but are rather one-sided. They primarily analyse the attention mechanisms or gradient-based attribution while neglecting the magnitudes of input feature values or the skip-connection module. Moreover, they inevitably bring spurious noisy pixel attributions unrelated to the model's decision, hindering humans' trust in the spotted visualization result. Hence, we propose an easy-to-implement but effective way to remedy this flaw: Smooth Noise Norm Attention (SNNA). We weigh the attention by the norm of the transformed value vector and guide the label-specific signal with the attention gradient, then randomly sample the input perturbations and average the corresponding gradients to produce noise-free attribution. Instead of evaluating the explanation method on the binary or multi-class classification tasks like in previous works, we explore the more complex multi-label classification scenario in this work, i.e., the driving action prediction task, and trained a model for it specifically. Both qualitative and quantitative evaluation results show the superiority of SNNA compared to other SOTA attention-based explainable methods in generating a clearer visual explanation map and ranking the input pixel importance.