Wide & Deep Learning for Node Classification

TL;DR

This paper introduces GCNIII, a flexible graph neural network framework that combines Wide & Deep architecture with novel techniques, improving node classification performance and addressing issues like heterophily and overfitting.

Contribution

It proposes GCNIII, integrating Wide & Deep architecture with three techniques, and explores LLMs for node feature engineering to enhance classification across domains.

Findings

GCNIII outperforms existing models on various tasks.

The framework effectively balances over-fitting and over-generalization.

Using LLMs improves cross-domain node classification.

Abstract

Wide & Deep, a simple yet effective learning architecture for recommendation systems developed by Google, has had a significant impact in both academia and industry due to its combination of the memorization ability of generalized linear models and the generalization ability of deep models. Graph convolutional networks (GCNs) remain dominant in node classification tasks; however, recent studies have highlighted issues such as heterophily and expressiveness, which focus on graph structure while seemingly neglecting the potential role of node features. In this paper, we propose a flexible framework GCNIII, which leverages the Wide & Deep architecture and incorporates three techniques: Intersect memory, Initial residual and Identity mapping. We provide comprehensive empirical evidence showing that GCNIII can more effectively balance the trade-off between over-fitting and over-generalization on various semi- and full- supervised tasks. Additionally, we explore the use of large language models (LLMs) for node feature engineering to enhance the performance of GCNIII in cross-domain node classification tasks. Our implementation is available at https://github.com/CYCUCAS/GCNIII.