Learning to Evolve: Bayesian-Guided Continual Knowledge Graph Embedding

TL;DR

This paper introduces BAKE, a Bayesian-guided continual knowledge graph embedding framework that effectively preserves prior knowledge and adapts to new information, outperforming existing methods in dynamic social media contexts.

Contribution

BAKE formulates CKGE as a Bayesian inference problem, providing a theoretically grounded, order-insensitive approach with a novel continual clustering regularization for semantic consistency.

Findings

BAKE achieves top performance on multiple CKGE benchmarks.

The Bayesian approach effectively mitigates catastrophic forgetting.

Continual clustering maintains semantic structure during evolution.

Abstract

As social media and the World Wide Web become hubs for information dissemination, effectively organizing and understanding the vast amounts of dynamically evolving Web content is crucial. Knowledge graphs (KGs) provide a powerful framework for structuring this information. However, the rapid emergence of new hot topics, user relationships, and events in social media renders traditional static knowledge graph embedding (KGE) models rapidly outdated. Continual Knowledge Graph Embedding (CKGE) aims to address this issue, but existing methods commonly suffer from catastrophic forgetting, whereby older, but still valuable, information is lost when learning new knowledge (such as new memes or trending events). This means the model cannot effectively learn the evolution of the data. We propose a novel CKGE framework, BAKE. Unlike existing methods, BAKE formulates CKGE as a sequential Bayesian inference problem and utilizes the Bayesian posterior update principle as a natural continual learning strategy. This principle is insensitive to data order and provides theoretical guarantees to preserve prior knowledge as much as possible. Specifically, we treat each batch of new data as a Bayesian update to the model's prior. By maintaining the posterior distribution, the model effectively preserves earlier knowledge even as it evolves over multiple snapshots. Furthermore, to constrain the evolution of knowledge across snapshots, we introduce a continual clustering method that maintains the compact cluster structure of entity embeddings through a regularization term, ensuring semantic consistency while allowing controlled adaptation to new knowledge. We conduct extensive experiments on multiple CKGE benchmarks, which demonstrate that BAKE achieves the top performance in the vast majority of cases compared to existing approaches.