Graph Relation Aware Continual Learning

TL;DR

This paper introduces RAM-CG, a relation-aware model for continual graph learning that leverages latent relations to improve knowledge transfer and task discrimination, outperforming existing methods on multiple datasets.

Contribution

The paper proposes a novel relation-aware adaptive model that explicitly explores latent relations and accounts for graph shifts, addressing limitations of prior methods.

Findings

RAM-CG achieves 2.2%, 6.9%, and 6.6% accuracy improvements on CitationNet, OGBN-arxiv, and TWITCH datasets.

Latent relations are effective invariant factors for evolving graphs.

Relation discovery enhances continual graph learning performance.

Abstract

Continual graph learning (CGL) studies the problem of learning from an infinite stream of graph data, consolidating historical knowledge, and generalizing it to the future task. At once, only current graph data are available. Although some recent attempts have been made to handle this task, we still face two potential challenges: 1) most of existing works only manipulate on the intermediate graph embedding and ignore intrinsic properties of graphs. It is non-trivial to differentiate the transferred information across graphs. 2) recent attempts take a parameter-sharing policy to transfer knowledge across time steps or progressively expand new architecture given shifted graph distribution. Learning a single model could loss discriminative information for each graph task while the model expansion scheme suffers from high model complexity. In this paper, we point out that latent relations behind graph edges can be attributed as an invariant factor for the evolving graphs and the statistical information of latent relations evolves. Motivated by this, we design a relation-aware adaptive model, dubbed as RAM-CG, that consists of a relation-discovery modular to explore latent relations behind edges and a task-awareness masking classifier to accounts for the shifted. Extensive experiments show that RAM-CG provides significant 2.2%, 6.9% and 6.6% accuracy improvements over the state-of-the-art results on CitationNet, OGBN-arxiv and TWITCH dataset, respective.