Multilingual Knowledge Editing with Language-Agnostic Factual Neurons

TL;DR

This paper introduces a novel multilingual knowledge editing method that leverages language-agnostic factual neurons in LLMs to improve editing accuracy and consistency across languages.

Contribution

The paper discovers language-agnostic factual neurons in LLMs and proposes LU-LAFNs, a method to edit multilingual knowledge by targeting these neurons, reducing conflicts and enhancing performance.

Findings

Achieves state-of-the-art edit performance on benchmarks

Identifies shared neurons representing multilingual facts

Demonstrates effectiveness of modeling semantic connections

Abstract

Multilingual knowledge editing (MKE) aims to simultaneously update factual knowledge across multiple languages within large language models (LLMs). Previous research indicates that the same knowledge across different languages within LLMs exhibits a degree of shareability. However, most existing MKE methods overlook the connections of the same knowledge between different languages, resulting in knowledge conflicts and limited edit performance. To address this issue, we first investigate how LLMs process multilingual factual knowledge and discover that the same factual knowledge in different languages generally activates a shared set of neurons, which we call language-agnostic factual neurons (LAFNs). These neurons represent the same factual knowledge shared across languages and imply the semantic connections among multilingual knowledge. Inspired by this finding, we propose a new MKE method by Locating and Updating Language-Agnostic Factual Neurons (LU-LAFNs) to edit multilingual knowledge simultaneously, which avoids knowledge conflicts and thus improves edit performance. Experimental results on Bi-ZsRE and MzsRE benchmarks demonstrate that our method achieves the best edit performance, indicating the effectiveness and importance of modeling the semantic connections among multilingual knowledge.