Self-Supervised Continual Graph Learning in Adaptive Riemannian Spaces

TL;DR

This paper introduces RieGrace, a self-supervised continual graph learning method in adaptive Riemannian spaces that dynamically models curvature and consolidates knowledge without labels, outperforming existing approaches.

Contribution

It proposes a novel self-supervised framework with adaptive Riemannian GCN and Lorentz distillation, addressing curvature variation and label scarcity in continual graph learning.

Findings

RieGrace outperforms baseline methods on benchmark datasets.

The adaptive Riemannian space effectively models curvature changes.

The Lorentz distillation improves knowledge consolidation without catastrophic forgetting.

Abstract

Continual graph learning routinely finds its role in a variety of real-world applications where the graph data with different tasks come sequentially. Despite the success of prior works, it still faces great challenges. On the one hand, existing methods work with the zero-curvature Euclidean space, and largely ignore the fact that curvature varies over the coming graph sequence. On the other hand, continual learners in the literature rely on abundant labels, but labeling graph in practice is particularly hard especially for the continuously emerging graphs on-the-fly. To address the aforementioned challenges, we propose to explore a challenging yet practical problem, the self-supervised continual graph learning in adaptive Riemannian spaces. In this paper, we propose a novel self-supervised Riemannian Graph Continual Learner (RieGrace). In RieGrace, we first design an Adaptive Riemannian GCN (AdaRGCN), a unified GCN coupled with a neural curvature adapter, so that Riemannian space is shaped by the learnt curvature adaptive to each graph. Then, we present a Label-free Lorentz Distillation approach, in which we create teacher-student AdaRGCN for the graph sequence. The student successively performs intra-distillation from itself and inter-distillation from the teacher so as to consolidate knowledge without catastrophic forgetting. In particular, we propose a theoretically grounded Generalized Lorentz Projection for the contrastive distillation in Riemannian space. Extensive experiments on the benchmark datasets show the superiority of RieGrace, and additionally, we investigate on how curvature changes over the graph sequence.