GNN Explanations that do not Explain and How to find Them

TL;DR

This paper investigates the failure modes of self-explainable GNNs, revealing that explanations can be unfaithful despite optimal model performance, and proposes a new metric to detect such degenerate explanations.

Contribution

The work characterizes failure cases of SE-GNN explanations, demonstrates their potential malicious and natural emergence, and introduces a new faithfulness metric to identify unfaithful explanations.

Findings

Degenerate explanations can be unfaithful despite optimal model risk.

Most faithfulness metrics fail to detect these degenerate explanations.

A new metric reliably marks unfaithful explanations in various scenarios.

Abstract

Explanations provided by Self-explainable Graph Neural Networks (SE-GNNs) are fundamental for understanding the model's inner workings and for identifying potential misuse of sensitive attributes. Although recent works have highlighted that these explanations can be suboptimal and potentially misleading, a characterization of their failure cases is unavailable. In this work, we identify a critical failure of SE-GNN explanations: explanations can be unambiguously unrelated to how the SE-GNNs infer labels. We show that, on the one hand, many SE-GNNs can achieve optimal true risk while producing these degenerate explanations, and on the other, most faithfulness metrics can fail to identify these failure modes. Our empirical analysis reveals that degenerate explanations can be maliciously planted (allowing an attacker to hide the use of sensitive attributes) and can also emerge naturally, highlighting the need for reliable auditing. To address this, we introduce a novel faithfulness metric that reliably marks degenerate explanations as unfaithful, in both malicious and natural settings. Our code is available at https://github.com/steveazzolin/gnn_deg_expl.