Semantic Interpretation and Validation of Graph Attention-based Explanations for GNN Models

TL;DR

This paper develops a semantic attention-based methodology to improve the explainability of GNN models, validating attention weights as feature importance indicators through semantic perturbations and applying it to lidar pointcloud data.

Contribution

It introduces a novel semantic validation approach for attention-based explanations in GNNs, extending explainability methods with semantic perturbations and analysis.

Findings

Attention weights correlate with model accuracy on semantic features.

Semantic perturbations reveal importance of key features.

Method successfully identifies influential semantic classes in lidar data.

Abstract

In this work, we propose a methodology for investigating the use of semantic attention to enhance the explainability of Graph Neural Network (GNN)-based models. Graph Deep Learning (GDL) has emerged as a promising field for tasks like scene interpretation, leveraging flexible graph structures to concisely describe complex features and relationships. As traditional explainability methods used in eXplainable AI (XAI) cannot be directly applied to such structures, graph-specific approaches are introduced. Attention has been previously employed to estimate the importance of input features in GDL, however, the fidelity of this method in generating accurate and consistent explanations has been questioned. To evaluate the validity of using attention weights as feature importance indicators, we introduce semantically-informed perturbations and correlate predicted attention weights with the accuracy of the model. Our work extends existing attention-based graph explainability methods by analysing the divergence in the attention distributions in relation to semantically sorted feature sets and the behaviour of a GNN model, efficiently estimating feature importance. We apply our methodology on a lidar pointcloud estimation model successfully identifying key semantic classes that contribute to enhanced performance, effectively generating reliable post-hoc semantic explanations.