Designing the Topology of Graph Neural Networks: A Novel Feature Fusion Perspective

TL;DR

This paper introduces F$^2$GNN, a novel framework for designing GNN topologies through feature fusion, which unifies existing methods and improves model capacity while addressing over-smoothing.

Contribution

The paper proposes a new feature fusion perspective for GNN topology design, unifying existing approaches and employing neural architecture search for optimized structures.

Findings

F$^2$GNN outperforms existing GNNs on eight real-world datasets.

The method alleviates over-smoothing by adaptively utilizing features at different levels.

Neural architecture search enhances the effectiveness of topology design.

Abstract

In recent years, Graph Neural Networks (GNNs) have shown superior performance on diverse real-world applications. To improve the model capacity, besides designing aggregation operations, GNN topology design is also very important. In general, there are two mainstream GNN topology design manners. The first one is to stack aggregation operations to obtain the higher-level features but easily got performance drop as the network goes deeper. Secondly, the multiple aggregation operations are utilized in each layer which provides adequate and independent feature extraction stage on local neighbors while are costly to obtain the higher-level information. To enjoy the benefits while alleviating the corresponding deficiencies of these two manners, we learn to design the topology of GNNs in a novel feature fusion perspective which is dubbed F$^2$GNN. To be specific, we provide a feature fusion perspective in designing GNN topology and propose a novel framework to unify the existing topology designs with feature selection and fusion strategies. Then we develop a neural architecture search method on top of the unified framework which contains a set of selection and fusion operations in the search space and an improved differentiable search algorithm. The performance gains on eight real-world datasets demonstrate the effectiveness of F$^2$GNN. We further conduct experiments to show that F$^2$GNN can improve the model capacity while alleviating the deficiencies of existing GNN topology design manners, especially alleviating the over-smoothing problem, by utilizing different levels of features adaptively.