Segmented Graph-Bert for Graph Instance Modeling

TL;DR

This paper introduces SEG-BERT, a segmented architecture adaptation of GRAPH-BERT, designed for graph instance representation learning, effectively handling variable sizes and orderless properties of graphs, and demonstrating superior performance on multiple benchmarks.

Contribution

The paper proposes SEG-BERT, a novel segmented architecture for GRAPH-BERT, tailored for graph instance learning, with strategies for size unification and unsupervised pre-training.

Findings

Outperforms comparison methods on six out of seven benchmark datasets.

Handles variable graph sizes and orderless properties effectively.

Pre-trainable in an unsupervised manner for transfer learning.

Abstract

In graph instance representation learning, both the diverse graph instance sizes and the graph node orderless property have been the major obstacles that render existing representation learning models fail to work. In this paper, we will examine the effectiveness of GRAPH-BERT on graph instance representation learning, which was designed for node representation learning tasks originally. To adapt GRAPH-BERT to the new problem settings, we re-design it with a segmented architecture instead, which is also named as SEG-BERT (Segmented GRAPH-BERT) for reference simplicity in this paper. SEG-BERT involves no node-order-variant inputs or functional components anymore, and it can handle the graph node orderless property naturally. What's more, SEG-BERT has a segmented architecture and introduces three different strategies to unify the graph instance sizes, i.e., full-input, padding/pruning and segment shifting, respectively. SEG-BERT is pre-trainable in an unsupervised manner, which can be further transferred to new tasks directly or with necessary fine-tuning. We have tested the effectiveness of SEG-BERT with experiments on seven graph instance benchmark datasets, and SEG-BERT can out-perform the comparison methods on six out of them with significant performance advantages.