Elastic Weight Consolidation for Knowledge Graph Continual Learning: An Empirical Evaluation

TL;DR

This paper evaluates Elastic Weight Consolidation (EWC) for mitigating catastrophic forgetting in knowledge graph link prediction tasks, showing significant reduction in forgetting with implications for continual learning protocols.

Contribution

It provides an empirical assessment of EWC's effectiveness in KG continual learning and highlights how task partitioning strategies influence forgetting.

Findings

EWC reduces catastrophic forgetting from 12.62% to 6.85%.

Task partitioning affects forgetting magnitude, with relation-based partitioning causing more forgetting.

Results demonstrate EWC's effectiveness in KG continual learning and emphasize evaluation protocol importance.

Abstract

Knowledge graphs (KGs) require continual updates as new information emerges, but neural embedding models suffer from catastrophic forgetting when learning new tasks sequentially. We evaluate Elastic Weight Consolidation (EWC), a regularization-based continual learning method, on KG link prediction using TransE embeddings on FB15k-237. Across multiple experiments with five random seeds, we find that EWC reduces catastrophic forgetting from 12.62% to 6.85%, a 45.7% reduction compared to naive sequential training. We observe that the task partitioning strategy affects the magnitude of forgetting: relation-based partitioning (grouping triples by relation type) exhibits 9.8 percentage points higher forgetting than randomly partitioned tasks (12.62% vs 2.81%), suggesting that task construction influences evaluation outcomes. While focused on a single embedding model and dataset, our results demonstrate that EWC effectively mitigates catastrophic forgetting in KG continual learning and highlight the importance of evaluation protocol design.