Language Anisotropic Cross-Lingual Model Editing

TL;DR

This paper introduces a novel framework for cross-lingual model editing in multilingual language models, enabling targeted updates to specific inputs across multiple languages using parallel corpora and anisotropic parameter adjustments.

Contribution

It defines the cross-lingual model editing task, proposes a framework for adapting monolingual editing to multiple languages, and introduces language anisotropic editing to enhance transferability.

Findings

Monolingual editing approaches fail to propagate edits across languages.

The proposed language anisotropic editing significantly improves cross-lingual transfer.

Empirical results demonstrate the effectiveness of the proposed method.

Abstract

Multilingual pre-trained language models can learn task-specific abilities or memorize facts across multiple languages but inevitably make undesired predictions with specific inputs. Under similar observation, model editing aims to post-hoc calibrate a model targeted to specific inputs with keeping the model's raw behavior. However, existing work only studies the monolingual scenario, which lacks the cross-lingual transferability to perform editing simultaneously across languages. In this work, we focus on cross-lingual model editing. Firstly, we define the cross-lingual model editing task and corresponding metrics, where an edit in one language propagates to the others. Next, we propose a framework to naturally adapt monolingual model editing approaches to the cross-lingual scenario using parallel corpus. Further, we propose language anisotropic editing to improve cross-lingual editing by amplifying different subsets of parameters for each language. On the newly defined cross-lingual model editing task, we empirically demonstrate the failure of monolingual baselines in propagating the edit to multiple languages and the effectiveness of the proposed language anisotropic model editing. Our code is publicly available at https://github.com/franklear/LiME.