Circle Feature Graphormer: Can Circle Features Stimulate Graph Transformer?

TL;DR

This paper introduces Circle Features to enhance graph transformer models for missing link prediction, achieving state-of-the-art results on the ogbl-citation2 dataset by incorporating local circle-based features as biases.

Contribution

The paper proposes novel Circle Features derived from social circle concepts and integrates them as biases into a graph transformer, improving link prediction performance.

Findings

CFG outperforms previous models on ogbl-citation2

Circle Features effectively enhance self-attention in graph transformers

The double tower structure captures both local and global features

Abstract

In this paper, we introduce two local graph features for missing link prediction tasks on ogbl-citation2. We define the features as Circle Features, which are borrowed from the concept of circle of friends. We propose the detailed computing formulas for the above features. Firstly, we define the first circle feature as modified swing for common graph, which comes from bipartite graph. Secondly, we define the second circle feature as bridge, which indicates the importance of two nodes for different circle of friends. In addition, we firstly propose the above features as bias to enhance graph transformer neural network, such that graph self-attention mechanism can be improved. We implement a Circled Feature aware Graph transformer (CFG) model based on SIEG network, which utilizes a double tower structure to capture both global and local structure features. Experimental results show that CFG achieves the state-of-the-art performance on dataset ogbl-citation2.