Cross-Lingual Activation Steering for Multilingual Language Models

TL;DR

This paper introduces Cross-Lingual Activation Steering (CLAS), a training-free method that improves multilingual model performance by selectively modulating neuron activations during inference, revealing latent capacities without changing model weights.

Contribution

The paper presents CLAS, a novel inference-time intervention that enhances multilingual model performance by activation modulation, without requiring retraining or fine-tuning.

Findings

CLAS improves classification accuracy by 2.3% and F1 score by 3.4%.

Performance correlates with increased language cluster separation.

Activation steering reveals latent multilingual capacities.

Abstract

Large language models exhibit strong multilingual capabilities, yet significant performance gaps persist between dominant and non-dominant languages. Prior work attributes this gap to imbalances between shared and language-specific neurons in multilingual representations. We propose Cross-Lingual Activation Steering (CLAS), a training-free inference-time intervention that selectively modulates neuron activations. We evaluate CLAS on classification and generation benchmarks, achieving average improvements of 2.3% (Acc.) and 3.4% (F1) respectively, while maintaining high-resource language performance. We discover that effective transfer operates through functional divergence rather than strict alignment; performance gains correlate with increased language cluster separation. Our results demonstrate that targeted activation steering can unlock latent multilingual capacity in existing models without modification to model weights.