Similarity-based Neighbor Selection for Graph LLMs

TL;DR

This paper introduces Similarity-based Neighbor Selection (SNS), a novel, training-free method that enhances graph representation in Text-attributed Graphs by improving neighbor quality, thereby boosting LLM-based node classification performance.

Contribution

The paper proposes SNS, a new neighbor selection technique that leverages SimCSE and advanced methods to improve graph learning with LLMs, addressing over-squashing and heterophily issues.

Findings

SNS outperforms vanilla GNNs on PubMed dataset.

SNS demonstrates superior scalability and generalization.

State-of-the-art results achieved in node classification.

Abstract

Text-attributed graphs (TAGs) present unique challenges for direct processing by Language Learning Models (LLMs), yet their extensive commonsense knowledge and robust reasoning capabilities offer great promise for node classification in TAGs. Prior research in this field has grappled with issues such as over-squashing, heterophily, and ineffective graph information integration, further compounded by inconsistencies in dataset partitioning and underutilization of advanced LLMs. To address these challenges, we introduce Similarity-based Neighbor Selection (SNS). Using SimCSE and advanced neighbor selection techniques, SNS effectively improves the quality of selected neighbors, thereby improving graph representation and alleviating issues like over-squashing and heterophily. Besides, as an inductive and training-free approach, SNS demonstrates superior generalization and scalability over traditional GNN methods. Our comprehensive experiments, adhering to standard dataset partitioning practices, demonstrate that SNS, through simple prompt interactions with LLMs, consistently outperforms vanilla GNNs and achieves state-of-the-art results on datasets like PubMed in node classification, showcasing LLMs' potential in graph structure understanding. Our research further underscores the significance of graph structure integration in LLM applications and identifies key factors for their success in node classification. Code is available at https://github.com/ruili33/SNS.