NTFormer: A Composite Node Tokenized Graph Transformer for Node Classification

TL;DR

NTFormer introduces a flexible token generation method for graph Transformers, enabling comprehensive node feature expression without graph-specific modifications, leading to improved node classification performance across diverse datasets.

Contribution

The paper presents NTFormer with Node2Par, a novel token generator that captures diverse graph features, enhancing Transformer-based node classification without requiring graph-specific adjustments.

Findings

Outperforms existing graph Transformers and GNNs on benchmark datasets

Effective on both homophily and heterophily graphs

Achieves superior accuracy across various graph scales

Abstract

Recently, the emerging graph Transformers have made significant advancements for node classification on graphs. In most graph Transformers, a crucial step involves transforming the input graph into token sequences as the model input, enabling Transformer to effectively learn the node representations. However, we observe that existing methods only express partial graph information of nodes through single-type token generation. Consequently, they require tailored strategies to encode additional graph-specific features into the Transformer to ensure the quality of node representation learning, limiting the model flexibility to handle diverse graphs. To this end, we propose a new graph Transformer called NTFormer to address this issue. NTFormer introduces a novel token generator called Node2Par, which constructs various token sequences using different token elements for each node. This flexibility allows Node2Par to generate valuable token sequences from different perspectives, ensuring comprehensive expression of rich graph features. Benefiting from the merits of Node2Par, NTFormer only leverages a Transformer-based backbone without graph-specific modifications to learn node representations, eliminating the need for graph-specific modifications. Extensive experiments conducted on various benchmark datasets containing homophily and heterophily graphs with different scales demonstrate the superiority of NTFormer over representative graph Transformers and graph neural networks for node classification.