ReDiSC: A Reparameterized Masked Diffusion Model for Scalable Node Classification with Structured Predictions

TL;DR

ReDiSC introduces a reparameterized masked diffusion model that estimates joint node label distributions, enabling scalable and effective structured node classification in graphs, outperforming existing methods especially on large datasets.

Contribution

The paper proposes ReDiSC, a novel reparameterized diffusion model for structured node classification, with theoretical efficiency advantages and practical scalability over prior diffusion-based approaches.

Findings

ReDiSC achieves superior or competitive accuracy on various graph datasets.

It scales effectively to large graphs where previous methods struggle.

ReDiSC outperforms state-of-the-art GNNs, label propagation, and diffusion baselines.

Abstract

In recent years, graph neural networks (GNN) have achieved unprecedented successes in node classification tasks. Although GNNs inherently encode specific inductive biases (e.g., acting as low-pass or high-pass filters), most existing methods implicitly assume conditional independence among node labels in their optimization objectives. While this assumption is suitable for traditional classification tasks such as image recognition, it contradicts the intuitive observation that node labels in graphs remain correlated, even after conditioning on the graph structure. To make structured predictions for node labels, we propose ReDiSC, namely, Reparameterized masked Diffusion model for Structured node Classification. ReDiSC estimates the joint distribution of node labels using a reparameterized masked diffusion model, which is learned through the variational expectation-maximization (EM) framework. Our theoretical analysis shows the efficiency advantage of ReDiSC in the E-step compared to DPM-SNC, a state-of-the-art model that relies on a manifold-constrained diffusion model in continuous domain. Meanwhile, we explicitly link ReDiSC's M-step objective to popular GNN and label propagation hybrid approaches. Extensive experiments demonstrate that ReDiSC achieves superior or highly competitive performance compared to state-of-the-art GNN, label propagation, and diffusion-based baselines across both homophilic and heterophilic graphs of varying sizes. Notably, ReDiSC scales effectively to large-scale datasets on which previous structured diffusion methods fail due to computational constraints, highlighting its significant practical advantage in structured node classification tasks.