Cross-Lingual Multi-Hop Knowledge Editing

TL;DR

This paper introduces a cross-lingual benchmark and a novel knowledge editing system for large language models, addressing performance gaps in multilingual settings and demonstrating significant improvements over prior methods.

Contribution

It proposes the first cross-lingual multi-hop knowledge editing framework, benchmark, and techniques that significantly enhance editing performance across multiple languages.

Findings

CLEVER-CKE achieves up to 30% improvement over prior methods.

Significant performance gaps exist between cross-lingual and English-centric editing.

The new benchmark CROLIN-MQUAKE effectively measures cross-lingual knowledge editing capabilities.

Abstract

Large language models are often expected to constantly adapt to new sources of knowledge and knowledge editing techniques aim to efficiently patch the outdated model knowledge, with minimal modification. Most prior works focus on monolingual knowledge editing in English, even though new information can emerge in any language from any part of the world. We propose the Cross-Lingual Multi-Hop Knowledge Editing paradigm, for measuring and analyzing the performance of various SoTA knowledge editing techniques in a cross-lingual setup. Specifically, we create a parallel cross-lingual benchmark, CROLIN-MQUAKE for measuring the knowledge editing capabilities. Our extensive analysis over various knowledge editing techniques uncover significant gaps in performance between the cross-lingual and English-centric setting. Following this, we propose a significantly improved system for cross-lingual multi-hop knowledge editing, CLEVER-CKE. CLEVER-CKE is based on a retrieve, verify and generate knowledge editing framework, where a retriever is formulated to recall edited facts and support an LLM to adhere to knowledge edits. We develop language-aware and hard-negative based contrastive objectives for improving the cross-lingual and fine-grained fact retrieval and verification process used in this framework. Extensive experiments on three LLMs, eight languages, and two datasets show CLEVER-CKE's significant gains of up to 30% over prior methods.