TL;DR
FLEKE introduces a federated approach to knowledge editing in large language models, enabling multi-client collaboration while preserving privacy and reducing redundant computations.
Contribution
The paper proposes FedEdit, a novel federated framework for LEKE that improves efficiency and privacy in multi-client knowledge editing of LLMs.
Findings
FedEdit retains over 96% of non-federated LEKE performance.
FedEdit outperforms FedAvg baseline by approximately twofold.
MEMIT is more consistent than PMET within the FLEKE framework.
Abstract
Locate-then-Edit Knowledge Editing (LEKE) is a key technique for updating large language models (LLMs) without full retraining. However, existing methods assume a single-user setting and become inefficient in real-world multi-client scenarios, where decentralized organizations (e.g., hospitals, financial institutions) independently update overlapping knowledge, leading to redundant mediator knowledge vector (MKV) computations and privacy concerns. To address these challenges, we introduce Federated Locate-then-Edit Knowledge Editing (FLEKE), a novel task that enables multiple clients to collaboratively perform LEKE while preserving privacy and reducing computational overhead. To achieve this, we propose FedEdit, a two-stage framework that optimizes MKV selection and reuse. In the first stage, clients locally apply LEKE and upload the computed MKVs. In the second stage, rather than…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
