Lens: Rethinking Multilingual Enhancement for Large Language Models

TL;DR

Lens introduces a novel method leveraging internal language representation spaces to enhance multilingual capabilities of large language models, achieving better performance with less resource expenditure.

Contribution

It proposes a new approach that improves multilingual abilities by manipulating internal representations, avoiding high costs and issues of existing data-driven methods.

Findings

Significantly improves multilingual performance

Maintains English proficiency effectively

Reduces computational cost compared to existing methods

Abstract

As global demand for multilingual large language models (LLMs) grows, most LLMs still remain overly focused on English, leading to the limited access to advanced AI for non-English speakers. Current methods to enhance multilingual capabilities largely rely on data-driven post-training techniques, such as multilingual instruction tuning or continual pre-training. However, these approaches exhibit significant limitations, including high resource cost, exacerbation of off-target issue and catastrophic forgetting of central language abilities. To this end, we propose Lens, a novel approach that enhances multilingual capabilities by leveraging LLMs' internal language representation spaces. Lens operates on two subspaces: the language-agnostic subspace, where it aligns target languages with the central language to inherit strong semantic representations, and the language-specific subspace, where it separates target and central languages to preserve linguistic specificity. Experiments on three English-centric LLMs show that Lens significantly improves multilingual performance while maintaining the model's English proficiency, achieving better results with less computational cost compared to existing post-training approaches.