Graph Neural Architecture Search with GPT-4

TL;DR

This paper introduces GNAS-LLM, a novel approach integrating GPT-4 into graph neural architecture search, reducing human effort and improving accuracy on benchmark datasets.

Contribution

It presents a new GNAS method using LLM prompts to guide architecture search, achieving better results with less human intervention.

Findings

Outperforms state-of-the-art GNAS methods on four benchmarks.

Achieves 0.7% and 0.3% improvements on validation and test sets.

Enhances search efficiency with fast convergence.

Abstract

Graph Neural Architecture Search (GNAS) has shown promising results in finding the best graph neural network architecture on a given graph dataset. However, existing GNAS methods still require intensive human labor and rich domain knowledge when designing the search space and search strategy. To this end, we integrate Large Language Models (LLMs) into GNAS and present a new GNAS model based on LLMs (GNAS-LLM for short). The basic idea of GNAS-LLM is to design a new class of GNAS prompts for LLMs to guide LLMs towards understanding the generative task of graph neural architectures. The prompts consist of descriptions of the search space, search strategy, and search feedback of GNAS. By iteratively running LLMs with the prompts, GNAS-LLM generates more accurate graph neural network architectures with fast convergence. Experimental results show that GNAS-LLM outperforms the state-of-the-art GNAS methods on four benchmark graph datasets, with an average improvement of 0.7% on the validation sets and 0.3% on the test sets. Besides, GNAS-LLM achieves an average improvement of 1.0% on the test sets based on the search space from AutoGEL.