CLaS-Bench: A Cross-Lingual Alignment and Steering Benchmark

TL;DR

CLaS-Bench is a new benchmark for evaluating multilingual steering techniques in large language models, enabling systematic assessment across 32 languages and revealing insights into language-specific structures and effective steering methods.

Contribution

It introduces CLaS-Bench, the first standardized benchmark for multilingual steering, and provides comprehensive evaluation of various steering techniques across multiple languages.

Findings

Residual-based DiffMean method outperforms others

Language-specific structures emerge in later layers

Steering directions cluster by language family

Abstract

Understanding and controlling the behavior of large language models (LLMs) is an increasingly important topic in multilingual NLP. Beyond prompting or fine-tuning, , i.e.,~manipulating internal representations during inference, has emerged as a more efficient and interpretable technique for adapting models to a target language. Yet, no dedicated benchmarks or evaluation protocols exist to quantify the effectiveness of steering techniques. We introduce CLaS-Bench, a lightweight parallel-question benchmark for evaluating language-forcing behavior in LLMs across 32 languages, enabling systematic evaluation of multilingual steering methods. We evaluate a broad array of steering techniques, including residual-stream DiffMean interventions, probe-derived directions, language-specific neurons, PCA/LDA vectors, Sparse Autoencoders, and prompting baselines. Steering performance is measured along two axes: language control and semantic relevance, combined into a single harmonic-mean steering score. We find that across languages simple residual-based DiffMean method consistently outperforms all other methods. Moreover, a layer-wise analysis reveals that language-specific structure emerges predominantly in later layers and steering directions cluster based on language family. CLaS-Bench is the first standardized benchmark for multilingual steering, enabling both rigorous scientific analysis of language representations and practical evaluation of steering as a low-cost adaptation alternative.