PolyFormer: Scalable Node-wise Filters via Polynomial Graph Transformer

TL;DR

PolyFormer introduces a scalable, attention-based approach for learning flexible node-wise filters in spectral graph neural networks, achieving high performance on large-scale graphs while balancing scalability and expressiveness.

Contribution

The paper proposes PolyAttn and PolyFormer, novel models that enable efficient, node-wise filter learning in spectral GNNs, overcoming scalability issues of previous methods.

Findings

Outperforms existing methods on various graph types.

Handles graphs with up to 100 million nodes.

Balances scalability and expressiveness effectively.

Abstract

Spectral Graph Neural Networks have demonstrated superior performance in graph representation learning. However, many current methods focus on employing shared polynomial coefficients for all nodes, i.e., learning node-unified filters, which limits the filters' flexibility for node-level tasks. The recent DSF attempts to overcome this limitation by learning node-wise coefficients based on positional encoding. However, the initialization and updating process of the positional encoding are burdensome, hindering scalability on large-scale graphs. In this work, we propose a scalable node-wise filter, PolyAttn. Leveraging the attention mechanism, PolyAttn can directly learn node-wise filters in an efficient manner, offering powerful representation capabilities. Building on PolyAttn, we introduce the whole model, named PolyFormer. In the lens of Graph Transformer models, PolyFormer, which calculates attention scores within nodes, shows great scalability. Moreover, the model captures spectral information, enhancing expressiveness while maintaining efficiency. With these advantages, PolyFormer offers a desirable balance between scalability and expressiveness for node-level tasks. Extensive experiments demonstrate that our proposed methods excel at learning arbitrary node-wise filters, showing superior performance on both homophilic and heterophilic graphs, and handling graphs containing up to 100 million nodes. The code is available at https://github.com/air029/PolyFormer.