Bi-Directional Multi-Scale Graph Dataset Condensation via Information Bottleneck

TL;DR

This paper introduces a bi-directional multi-scale graph dataset condensation framework that preserves maximum information across scales, improving efficiency and stability in graph data compression for diverse on-device scenarios.

Contribution

It proposes a novel GNN-centric bi-directional condensation method based on mutual information theory and eigenbasis matching, unifying large-to-small and small-to-large scale paradigms.

Findings

Outperforms existing methods in graph condensation across multiple datasets.

Achieves stable and consistent multi-scale graph compression.

Effectively preserves original graph information at different scales.

Abstract

Dataset condensation has significantly improved model training efficiency, but its application on devices with different computing power brings new requirements for different data sizes. Thus, condensing multiple scale graphs simultaneously is the core of achieving efficient training in different on-device scenarios. Existing efficient works for multi-scale graph dataset condensation mainly perform efficient approximate computation in scale order (large-to-small or small-to-large scales). However, for non-Euclidean structures of sparse graph data, these two commonly used paradigms for multi-scale graph dataset condensation have serious scaling down degradation and scaling up collapse problems of a graph. The main bottleneck of the above paradigms is whether the effective information of the original graph is fully preserved when consenting to the primary sub-scale (the first of multiple scales), which determines the condensation effect and consistency of all scales. In this paper, we proposed a novel GNN-centric Bi-directional Multi-Scale Graph Dataset Condensation (BiMSGC) framework, to explore unifying paradigms by operating on both large-to-small and small-to-large for multi-scale graph condensation. Based on the mutual information theory, we estimate an optimal ``meso-scale'' to obtain the minimum necessary dense graph preserving the maximum utility information of the original graph, and then we achieve stable and consistent ``bi-directional'' condensation learning by optimizing graph eigenbasis matching with information bottleneck on other scales. Encouraging empirical results on several datasets demonstrates the significant superiority of the proposed framework in graph condensation at different scales.