Personalized PageRank Graph Attention Networks

TL;DR

This paper introduces a novel graph neural network model that integrates Personalized PageRank into graph attention mechanisms, enabling larger neighborhood information aggregation without over-smoothing, and demonstrates superior performance on benchmark datasets.

Contribution

It presents a new GNN architecture combining Personalized PageRank with attention mechanisms to effectively utilize larger neighborhoods without over-smoothing.

Findings

Outperforms baseline models on four benchmark datasets

Incorporates infinitely many neighborhood aggregations via Personalized PageRank

Publicly available implementation

Abstract

There has been a rising interest in graph neural networks (GNNs) for representation learning over the past few years. GNNs provide a general and efficient framework to learn from graph-structured data. However, GNNs typically only use the information of a very limited neighborhood for each node to avoid over-smoothing. A larger neighborhood would be desirable to provide the model with more information. In this work, we incorporate the limit distribution of Personalized PageRank (PPR) into graph attention networks (GATs) to reflect the larger neighbor information without introducing over-smoothing. Intuitively, message aggregation based on Personalized PageRank corresponds to infinitely many neighborhood aggregation layers. We show that our models outperform a variety of baseline models for four widely used benchmark datasets. Our implementation is publicly available online.