Learn Layer-wise Connections in Graph Neural Networks

TL;DR

This paper introduces LLC, a neural architecture search framework that learns adaptive layer-wise connections in GNNs, improving performance and reducing over-smoothing across diverse datasets.

Contribution

The paper proposes a novel NAS-based framework for automatically learning layer-wise connections in GNNs, addressing data-specific performance variations.

Findings

Improved GNN performance on multiple datasets

Reduced over-smoothing in GNNs

Effective search algorithm for layer connections

Abstract

In recent years, Graph Neural Networks (GNNs) have shown superior performance on diverse applications on real-world datasets. To improve the model capacity and alleviate the over-smoothing problem, several methods proposed to incorporate the intermediate layers by layer-wise connections. However, due to the highly diverse graph types, the performance of existing methods vary on diverse graphs, leading to a need for data-specific layer-wise connection methods. To address this problem, we propose a novel framework LLC (Learn Layer-wise Connections) based on neural architecture search (NAS) to learn adaptive connections among intermediate layers in GNNs. LLC contains one novel search space which consists of 3 types of blocks and learnable connections, and one differentiable search algorithm to enable the efficient search process. Extensive experiments on five real-world datasets are conducted, and the results show that the searched layer-wise connections can not only improve the performance but also alleviate the over-smoothing problem.