Layer-wise Swapping for Generalizable Multilingual Safety

TL;DR

This paper introduces a layer swapping technique that transfers safety alignment from English to low-resource languages in LLMs, improving safety without sacrificing general language understanding.

Contribution

It proposes a novel safety-aware layer swapping method that adaptively transfers safety alignment across languages without additional training.

Findings

Achieves safety improvements on multilingual benchmarks

Maintains performance on general language understanding tasks

Produces more aligned and less harmful responses

Abstract

Despite the rapid advancements of Large Language Models (LLMs), safety risks remain a critical challenge for low-resource languages. Existing safety datasets are predominantly English centric, limiting progress in multilingual safety alignment. As a result, low resource expert models, finetuned on their respective instruction datasets, tend to exhibit higher unsafety rates compared to their high resource counterparts. In this work, we propose a safety aware layer swapping method that transfers safety alignment from an English safety expert to low resource language experts without additional training. To further enhance transfer ability, our method adaptively selects or blends modules based on their degree of specialization. Our approach preserves performance on general language understanding tasks while enhancing safety in the target languages. Experimental results show that the proposed method achieves comparable performance to the language expert on general benchmarks such as MMMLU, BELEBELE, and MGSM, while producing more aligned and less harmful responses on the MultiJail safety benchmark.