TAGA: Text-Attributed Graph Self-Supervised Learning by Synergizing Graph and Text Mutual Transformations

TL;DR

This paper presents TAGA, a self-supervised learning framework for Text-Attributed Graphs that aligns structural and semantic views to improve representation learning without extensive labeled data.

Contribution

Introducing TAGA, a novel self-supervised framework that synergizes graph and text views for TAGs, enabling effective learning in low-label scenarios.

Findings

Strong performance in zero-shot scenarios

Effective across eight real-world datasets

Outperforms supervised methods in limited data settings

Abstract

Text-Attributed Graphs (TAGs) enhance graph structures with natural language descriptions, enabling detailed representation of data and their relationships across a broad spectrum of real-world scenarios. Despite the potential for deeper insights, existing TAG representation learning primarily relies on supervised methods, necessitating extensive labeled data and limiting applicability across diverse contexts. This paper introduces a new self-supervised learning framework, Text-And-Graph Multi-View Alignment (TAGA), which overcomes these constraints by integrating TAGs' structural and semantic dimensions. TAGA constructs two complementary views: Text-of-Graph view, which organizes node texts into structured documents based on graph topology, and the Graph-of-Text view, which converts textual nodes and connections into graph data. By aligning representations from both views, TAGA captures joint textual and structural information. In addition, a novel structure-preserving random walk algorithm is proposed for efficient training on large-sized TAGs. Our framework demonstrates strong performance in zero-shot and few-shot scenarios across eight real-world datasets.