Auto-GNN: Neural Architecture Search of Graph Neural Networks

TL;DR

This paper introduces AGNN, a neural architecture search framework tailored for graph neural networks, which efficiently finds optimal architectures and outperforms existing models on benchmark datasets.

Contribution

The paper proposes a novel NAS framework specifically designed for GNNs, including a reinforcement learning controller and a new parameter sharing strategy.

Findings

AGNN achieves state-of-the-art performance on benchmark datasets.

The proposed parameter sharing strategy stabilizes GNN architecture search.

AGNN outperforms handcrafted GNN models and traditional NAS methods.

Abstract

Graph neural networks (GNN) has been successfully applied to operate on the graph-structured data. Given a specific scenario, rich human expertise and tremendous laborious trials are usually required to identify a suitable GNN architecture. It is because the performance of a GNN architecture is significantly affected by the choice of graph convolution components, such as aggregate function and hidden dimension. Neural architecture search (NAS) has shown its potential in discovering effective deep architectures for learning tasks in image and language modeling. However, existing NAS algorithms cannot be directly applied to the GNN search problem. First, the search space of GNN is different from the ones in existing NAS work. Second, the representation learning capacity of GNN architecture changes obviously with slight architecture modifications. It affects the search efficiency of traditional search methods. Third, widely used techniques in NAS such as parameter sharing might become unstable in GNN. To bridge the gap, we propose the automated graph neural networks (AGNN) framework, which aims to find an optimal GNN architecture within a predefined search space. A reinforcement learning based controller is designed to greedily validate architectures via small steps. AGNN has a novel parameter sharing strategy that enables homogeneous architectures to share parameters, based on a carefully-designed homogeneity definition. Experiments on real-world benchmark datasets demonstrate that the GNN architecture identified by AGNN achieves the best performance, comparing with existing handcrafted models and tradistional search methods.