Exploring the Potential of Large Language Models in Graph Generation

TL;DR

This paper investigates the capabilities of large language models, especially GPT-4, in generating graphs with specific properties, revealing their preliminary success and limitations in rule-based and distribution-based graph generation tasks.

Contribution

It systematically explores LLMs' ability to generate graphs, introduces tailored tasks, and evaluates their performance, highlighting potential and challenges in property-based graph generation.

Findings

GPT-4 shows preliminary graph generation abilities.

Few-shot and chain-of-thought prompting do not always improve results.

LLMs can generate molecules with specific properties.

Abstract

Large language models (LLMs) have achieved great success in many fields, and recent works have studied exploring LLMs for graph discriminative tasks such as node classification. However, the abilities of LLMs for graph generation remain unexplored in the literature. Graph generation requires the LLM to generate graphs with given properties, which has valuable real-world applications such as drug discovery, while tends to be more challenging. In this paper, we propose LLM4GraphGen to explore the ability of LLMs for graph generation with systematical task designs and extensive experiments. Specifically, we propose several tasks tailored with comprehensive experiments to address key questions regarding LLMs' understanding of different graph structure rules, their ability to capture structural type distributions, and their utilization of domain knowledge for property-based graph generation. Our evaluations demonstrate that LLMs, particularly GPT-4, exhibit preliminary abilities in graph generation tasks, including rule-based and distribution-based generation. We also observe that popular prompting methods, such as few-shot and chain-of-thought prompting, do not consistently enhance performance. Besides, LLMs show potential in generating molecules with specific properties. These findings may serve as foundations for designing good LLMs based models for graph generation and provide valuable insights and further research.