Massive Activations in Graph Neural Networks: Decoding Attention for Domain-Dependent Interpretability

TL;DR

This paper uncovers Massive Activations in attention-based GNNs, showing they encode domain-specific signals and can be used for interpretability, especially in molecular graphs.

Contribution

It introduces a novel method to detect Massive Activations in edge-featured GNNs and links these activations to domain-relevant information, enhancing interpretability.

Findings

Massive Activations correlate with common bond types in molecules

MAs can serve as attribution indicators for less informative edges

The method is validated on benchmark datasets like ZINC, TOX21, and PROTEINS.

Abstract

Graph Neural Networks (GNNs) have become increasingly popular for effectively modeling graph-structured data, and attention mechanisms have been pivotal in enabling these models to capture complex patterns. In our study, we reveal a critical yet underexplored consequence of integrating attention into edge-featured GNNs: the emergence of Massive Activations (MAs) within attention layers. By developing a novel method for detecting MAs on edge features, we show that these extreme activations are not only activation anomalies but encode domain-relevant signals. Our post-hoc interpretability analysis demonstrates that, in molecular graphs, MAs aggregate predominantly on common bond types (e.g., single and double bonds) while sparing more informative ones (e.g., triple bonds). Furthermore, our ablation studies confirm that MAs can serve as natural attribution indicators, reallocating to less informative edges. Our study assesses various edge-featured attention-based GNN models using benchmark datasets, including ZINC, TOX21, and PROTEINS. Key contributions include (1) establishing the direct link between attention mechanisms and MAs generation in edge-featured GNNs, (2) developing a robust definition and detection method for MAs enabling reliable post-hoc interpretability. Overall, our study reveals the complex interplay between attention mechanisms, edge-featured GNNs model, and MAs emergence, providing crucial insights for relating GNNs internals to domain knowledge.