Probabilistic Dual Network Architecture Search on Graphs

TL;DR

This paper introduces PDNAS, a differentiable NAS method for GNNs that optimizes both micro- and macro-architectures, leading to better performance across datasets.

Contribution

It presents the first fully gradient-based differentiable NAS framework for GNNs, optimizing both intra-block operations and inter-block connections.

Findings

PDNAS outperforms hand-designed GNNs and other NAS methods.

On the PPI dataset, PDNAS exceeds competitors by 1.67 and 0.17 in F1 scores.

It enables deeper GNN architectures with improved adaptability.

Abstract

We present the first differentiable Network Architecture Search (NAS) for Graph Neural Networks (GNNs). GNNs show promising performance on a wide range of tasks, but require a large amount of architecture engineering. First, graphs are inherently a non-Euclidean and sophisticated data structure, leading to poor adaptivity of GNN architectures across different datasets. Second, a typical graph block contains numerous different components, such as aggregation and attention, generating a large combinatorial search space. To counter these problems, we propose a Probabilistic Dual Network Architecture Search (PDNAS) framework for GNNs. PDNAS not only optimises the operations within a single graph block (micro-architecture), but also considers how these blocks should be connected to each other (macro-architecture). The dual architecture (micro- and marco-architectures) optimisation allows PDNAS to find deeper GNNs on diverse datasets with better performance compared to other graph NAS methods. Moreover, we use a fully gradient-based search approach to update architectural parameters, making it the first differentiable graph NAS method. PDNAS outperforms existing hand-designed GNNs and NAS results, for example, on the PPI dataset, PDNAS beats its best competitors by 1.67 and 0.17 in F1 scores.