Graph Structure Learning with Variational Information Bottleneck

TL;DR

This paper introduces VIB-GSL, a novel graph structure learning framework guided by the Variational Information Bottleneck, which improves robustness and effectiveness of GNNs by learning task-relevant, compressed graph structures.

Contribution

It extends the Information Bottleneck principle to graph structure learning, providing a universal framework for extracting task-relevant relations from noisy or incomplete graphs.

Findings

VIB-GSL outperforms existing methods in robustness and effectiveness.

It provides a stable training process via a variational IB objective.

Experimental results confirm superior performance across multiple tasks.

Abstract

Graph Neural Networks (GNNs) have shown promising results on a broad spectrum of applications. Most empirical studies of GNNs directly take the observed graph as input, assuming the observed structure perfectly depicts the accurate and complete relations between nodes. However, graphs in the real world are inevitably noisy or incomplete, which could even exacerbate the quality of graph representations. In this work, we propose a novel Variational Information Bottleneck guided Graph Structure Learning framework, namely VIB-GSL, in the perspective of information theory. VIB-GSL advances the Information Bottleneck (IB) principle for graph structure learning, providing a more elegant and universal framework for mining underlying task-relevant relations. VIB-GSL learns an informative and compressive graph structure to distill the actionable information for specific downstream tasks. VIB-GSL deduces a variational approximation for irregular graph data to form a tractable IB objective function, which facilitates training stability. Extensive experimental results demonstrate that the superior effectiveness and robustness of VIB-GSL.