Graph Contrastive Learning with Low-Rank Regularization and Low-Rank Attention for Noisy Node Classification

TL;DR

This paper introduces GCL-LRR, a robust graph contrastive learning method with low-rank regularization for noisy node classification, supported by theoretical insights and improved by an LR-Attention layer, demonstrating strong empirical results.

Contribution

The paper proposes a novel low-rank regularized contrastive learning framework for noisy graph data, with theoretical backing and an enhanced LR-Attention model for better performance.

Findings

GCL-LRR outperforms existing methods on benchmark datasets.

Theoretical analysis shows low-rank regularization improves generalization.

GCL-LR-Attention achieves tighter bounds and higher accuracy.

Abstract

Graph Neural Networks (GNNs) have achieved remarkable success in learning node representations and have shown strong performance in tasks such as node classification. However, recent findings indicate that the presence of noise in real-world graph data can substantially impair the effectiveness of GNNs. To address this challenge, we introduce a robust and innovative node representation learning method named Graph Contrastive Learning with Low-Rank Regularization, or GCL-LRR, which follows a two-stage transductive learning framework for node classification. In the first stage, the GCL-LRR encoder is optimized through prototypical contrastive learning while incorporating a low-rank regularization objective. In the second stage, the representations generated by GCL-LRR are employed by a linear transductive classifier to predict the labels of unlabeled nodes within the graph. Our GCL-LRR is inspired by the Low Frequency Property (LFP) of the graph data and its labels, and it is also theoretically motivated by our sharp generalization bound for transductive learning. To the best of our knowledge, our theoretical result is among the first to theoretically demonstrate the advantage of low-rank regularization in transductive learning, which is also supported by strong empirical results. To further enhance the performance of GCL-LRR, we present an improved model named GCL-LR-Attention, which incorporates a novel LR-Attention layer into GCL-LRR. GCL-LR-Attention reduces the kernel complexity of GCL-LRR and contributes to a tighter generalization bound, leading to improved performance. Extensive evaluations on standard benchmark datasets evidence the effectiveness and robustness of both GCL-LRR and GCL-LR-Attention in learning meaningful node representations. The code is available at https://github.com/Statistical-Deep-Learning/GCL-LR-Attention.