HERO: Heterogeneous Continual Graph Learning via Meta-Knowledge Distillation

TL;DR

HERO is a novel framework for continual learning on evolving heterogeneous graphs, combining meta-adaptation, diversity sampling, and heterogeneity-aware knowledge distillation to improve knowledge retention and adaptation.

Contribution

This work introduces HERO, integrating meta-learning, diversity sampling, and knowledge distillation specifically designed for dynamic heterogeneous graphs.

Findings

HERO significantly reduces catastrophic forgetting in experiments.

HERO achieves superior performance on web-related graph benchmarks.

The framework enables efficient knowledge reuse in evolving graph environments.

Abstract

Heterogeneous graph neural networks have seen rapid progress in web applications such as social networks, knowledge graphs, and recommendation systems, driven by the inherent heterogeneity of web data. However, existing methods typically assume static graphs, while real-world graphs are continuously evolving. This dynamic nature requires models to adapt to new data while preserving existing knowledge. To this end, this work introduces HERO (HEterogeneous continual gRaph learning via meta-knOwledge distillation), a unified framework for continual learning on heterogeneous graphs. HERO employs meta-adaptation, a gradient-based meta-learning strategy that provides directional guidance for rapid adaptation to new tasks with limited samples. To enable efficient and effective knowledge reuse, we propose DiSCo (Diversity Sampling with semantic Consistency), a heterogeneity-aware sampling method that maximizes target node diversity and expands subgraphs along metapaths, retaining critical semantic and structural information with minimal overhead. Furthermore, HERO incorporates heterogeneity-aware knowledge distillation, which aligns knowledge at both the node and semantic levels to balance adaptation and retention across tasks. Extensive experiments on four web-related heterogeneous graph benchmarks demonstrate that HERO substantially mitigates catastrophic forgetting while achieving efficient and consistent knowledge reuse in dynamic web environments.