MLGIB: Multi-Label Graph Information Bottleneck for Expressive and Robust Message Passing

TL;DR

MLGIB introduces a novel message-passing framework for multi-label graphs that enhances expressiveness and robustness by balancing label signal preservation with noise suppression.

Contribution

It formulates multi-label message passing as constrained information transmission, deriving variational bounds for an end-to-end label-aware architecture.

Findings

Consistent improvements over existing methods on multiple benchmarks.

Effectively balances expressiveness and robustness in multi-label graph learning.

Validates the generality of the proposed framework.

Abstract

Graph Neural Networks (GNNs) suffer from over-squashing in deep message passing, where information from exponentially growing neighborhoods is compressed into fixed-dimensional representations. We show that this issue becomes a distinct failure mode in multi-label graphs: neighboring nodes often share only limited labels while differing across many irrelevant ones, causing predictive signals to be diluted by noisy label information. To address this challenge, we propose the Multi-Label Graph Information Bottleneck (MLGIB), which formulates multi-label message passing as constrained information transmission under irrelevant label noise. MLGIB balances expressiveness and robustness by preserving predictive label signals while suppressing irrelevant noise. Specifically, it constructs a Markovian dependence space and derives tractable variational bounds, where the lower bound maximizes mutual information with target labels and the upper bound constrains redundant source information. These bounds lead to an end-to-end label-aware message-passing architecture. Extensive experiments on multiple benchmarks demonstrate consistent improvements over existing methods, validating the effectiveness and generality of the proposed framework.