A Graph is Worth $K$ Words: Euclideanizing Graph using Pure Transformer

TL;DR

This paper introduces GraphsGPT, a novel framework that transforms non-Euclidean graphs into Euclidean vectors using a Graph2Seq encoder and reconstructs them with a GraphGPT decoder, enabling effective graph representation and generation.

Contribution

It presents a new method for Euclideanizing graphs with a Graph2Seq and GraphGPT, achieving state-of-the-art results in graph tasks and enabling advanced graph generation and mixup.

Findings

State-of-the-art performance on 8/9 graph tasks

Strong ability in few-shot and conditional graph generation

Effective graph mixup in Euclidean space

Abstract

Can we model Non-Euclidean graphs as pure language or even Euclidean vectors while retaining their inherent information? The Non-Euclidean property have posed a long term challenge in graph modeling. Despite recent graph neural networks and graph transformers efforts encoding graphs as Euclidean vectors, recovering the original graph from vectors remains a challenge. In this paper, we introduce GraphsGPT, featuring an Graph2Seq encoder that transforms Non-Euclidean graphs into learnable Graph Words in the Euclidean space, along with a GraphGPT decoder that reconstructs the original graph from Graph Words to ensure information equivalence. We pretrain GraphsGPT on $100$M molecules and yield some interesting findings: (1) The pretrained Graph2Seq excels in graph representation learning, achieving state-of-the-art results on $8/9$ graph classification and regression tasks. (2) The pretrained GraphGPT serves as a strong graph generator, demonstrated by its strong ability to perform both few-shot and conditional graph generation. (3) Graph2Seq+GraphGPT enables effective graph mixup in the Euclidean space, overcoming previously known Non-Euclidean challenges. (4) The edge-centric pretraining framework GraphsGPT demonstrates its efficacy in graph domain tasks, excelling in both representation and generation. Code is available at \href{https://github.com/A4Bio/GraphsGPT}{GitHub}.