GraTO: Graph Neural Network Framework Tackling Over-smoothing with Neural Architecture Search

TL;DR

GraTO is a neural architecture search framework for GNNs that balances performance and over-smoothing, incorporating novel solutions like DropAttribute to improve node representation quality across multiple layers.

Contribution

Introduces GraTO, a framework that jointly leverages multiple over-smoothing solutions via neural architecture search, including a novel loss function and DropAttribute scheme.

Findings

Outperforms baselines in over-smoothing metrics

Achieves competitive accuracy on six datasets

Enhances robustness with deeper GNN layers

Abstract

Current Graph Neural Networks (GNNs) suffer from the over-smoothing problem, which results in indistinguishable node representations and low model performance with more GNN layers. Many methods have been put forward to tackle this problem in recent years. However, existing tackling over-smoothing methods emphasize model performance and neglect the over-smoothness of node representations. Additional, different approaches are applied one at a time, while there lacks an overall framework to jointly leverage multiple solutions to the over-smoothing challenge. To solve these problems, we propose GraTO, a framework based on neural architecture search to automatically search for GNNs architecture. GraTO adopts a novel loss function to facilitate striking a balance between model performance and representation smoothness. In addition to existing methods, our search space also includes DropAttribute, a novel scheme for alleviating the over-smoothing challenge, to fully leverage diverse solutions. We conduct extensive experiments on six real-world datasets to evaluate GraTo, which demonstrates that GraTo outperforms baselines in the over-smoothing metrics and achieves competitive performance in accuracy. GraTO is especially effective and robust with increasing numbers of GNN layers. Further experiments bear out the quality of node representations learned with GraTO and the effectiveness of model architecture. We make cide of GraTo available at Github (\url{https://github.com/fxsxjtu/GraTO}).