Communication-Efficient Federated Knowledge Graph Embedding with Entity-Wise Top-K Sparsification

TL;DR

This paper introduces FedS, a communication-efficient federated knowledge graph embedding method that selectively transmits only the most changed entity embeddings, significantly reducing communication costs without sacrificing performance.

Contribution

The paper proposes a novel Entity-Wise Top-K Sparsification strategy and an Intermittent Synchronization Mechanism to improve communication efficiency in federated knowledge graph embedding.

Findings

FedS reduces communication costs significantly.

Negligible performance degradation observed.

Effective in heterogeneous federated environments.

Abstract

Federated Knowledge Graphs Embedding learning (FKGE) encounters challenges in communication efficiency stemming from the considerable size of parameters and extensive communication rounds. However, existing FKGE methods only focus on reducing communication rounds by conducting multiple rounds of local training in each communication round, and ignore reducing the size of parameters transmitted within each communication round. To tackle the problem, we first find that universal reduction in embedding precision across all entities during compression can significantly impede convergence speed, underscoring the importance of maintaining embedding precision. We then propose bidirectional communication-efficient FedS based on Entity-Wise Top-K Sparsification strategy. During upload, clients dynamically identify and upload only the Top-K entity embeddings with the greater changes to the server. During download, the server first performs personalized embedding aggregation for each client. It then identifies and transmits the Top-K aggregated embeddings to each client. Besides, an Intermittent Synchronization Mechanism is used by FedS to mitigate negative effect of embedding inconsistency among shared entities of clients caused by heterogeneity of Federated Knowledge Graph. Extensive experiments across three datasets showcase that FedS significantly enhances communication efficiency with negligible (even no) performance degradation.