Heterogeneous Federated Knowledge Graph Embedding Learning and Unlearning

TL;DR

This paper introduces FedLU, a federated learning framework for heterogeneous knowledge graph embedding that enhances knowledge transfer, addresses data heterogeneity, and enables knowledge unlearning while preserving privacy.

Contribution

FedLU is the first framework to combine federated KG embedding with unlearning, using mutual knowledge distillation and neuroscience-inspired methods for effective knowledge management.

Findings

FedLU outperforms existing methods in link prediction tasks.

It effectively unlearns specific knowledge without degrading overall performance.

The framework handles data heterogeneity and knowledge forgetting efficiently.

Abstract

Federated Learning (FL) recently emerges as a paradigm to train a global machine learning model across distributed clients without sharing raw data. Knowledge Graph (KG) embedding represents KGs in a continuous vector space, serving as the backbone of many knowledge-driven applications. As a promising combination, federated KG embedding can fully take advantage of knowledge learned from different clients while preserving the privacy of local data. However, realistic problems such as data heterogeneity and knowledge forgetting still remain to be concerned. In this paper, we propose FedLU, a novel FL framework for heterogeneous KG embedding learning and unlearning. To cope with the drift between local optimization and global convergence caused by data heterogeneity, we propose mutual knowledge distillation to transfer local knowledge to global, and absorb global knowledge back. Moreover, we present an unlearning method based on cognitive neuroscience, which combines retroactive interference and passive decay to erase specific knowledge from local clients and propagate to the global model by reusing knowledge distillation. We construct new datasets for assessing realistic performance of the state-of-the-arts. Extensive experiments show that FedLU achieves superior results in both link prediction and knowledge forgetting.