AGALE: A Graph-Aware Continual Learning Evaluation Framework

TL;DR

The paper introduces AGALE, a comprehensive evaluation framework for continual learning on graph-structured data, addressing limitations of existing methods by supporting multi-label scenarios and analyzing the impact of graph properties.

Contribution

It develops a novel graph-aware evaluation framework for continual graph learning, including new settings, data partitioning algorithms, and theoretical insights on homophily effects.

Findings

AGALE supports multi-label node scenarios.

The framework reveals the impact of homophily on method performance.

Extensive experiments compare various continual learning approaches.

Abstract

In recent years, continual learning (CL) techniques have made significant progress in learning from streaming data while preserving knowledge across sequential tasks, particularly in the realm of euclidean data. To foster fair evaluation and recognize challenges in CL settings, several evaluation frameworks have been proposed, focusing mainly on the single- and multi-label classification task on euclidean data. However, these evaluation frameworks are not trivially applicable when the input data is graph-structured, as they do not consider the topological structure inherent in graphs. Existing continual graph learning (CGL) evaluation frameworks have predominantly focussed on single-label scenarios in the node classification (NC) task. This focus has overlooked the complexities of multi-label scenarios, where nodes may exhibit affiliations with multiple labels, simultaneously participating in multiple tasks. We develop a graph-aware evaluation (\agale) framework that accommodates both single-labeled and multi-labeled nodes, addressing the limitations of previous evaluation frameworks. In particular, we define new incremental settings and devise data partitioning algorithms tailored to CGL datasets. We perform extensive experiments comparing methods from the domains of continual learning, continual graph learning, and dynamic graph learning (DGL). We theoretically analyze \agale and provide new insights about the role of homophily in the performance of compared methods. We release our framework at https://github.com/Tianqi-py/AGALE.