How to Find Your Friendly Neighborhood: Graph Attention Design with Self-Supervision

TL;DR

This paper introduces SuperGAT, a self-supervised graph attention network that improves handling noisy graphs by leveraging edge prediction tasks, with guidance based on graph homophily and degree.

Contribution

The paper proposes a novel self-supervised attention mechanism for graph neural networks, enhancing robustness to noise and providing a design recipe based on graph characteristics.

Findings

SuperGAT outperforms baselines on 15 out of 17 datasets.

Graph homophily and average degree influence attention effectiveness.

The proposed method generalizes well across diverse real-world datasets.

Abstract

Attention mechanism in graph neural networks is designed to assign larger weights to important neighbor nodes for better representation. However, what graph attention learns is not understood well, particularly when graphs are noisy. In this paper, we propose a self-supervised graph attention network (SuperGAT), an improved graph attention model for noisy graphs. Specifically, we exploit two attention forms compatible with a self-supervised task to predict edges, whose presence and absence contain the inherent information about the importance of the relationships between nodes. By encoding edges, SuperGAT learns more expressive attention in distinguishing mislinked neighbors. We find two graph characteristics influence the effectiveness of attention forms and self-supervision: homophily and average degree. Thus, our recipe provides guidance on which attention design to use when those two graph characteristics are known. Our experiment on 17 real-world datasets demonstrates that our recipe generalizes across 15 datasets of them, and our models designed by recipe show improved performance over baselines.