Cross-Family Universality of Behavioral Axes via Anchor-Projected Representations

TL;DR

This paper introduces an anchor-projection framework that maps hidden representations from different large language models into a shared space, enabling cross-family behavioral direction transfer without fine-tuning.

Contribution

The authors propose a novel anchor-projection method that aligns behavioral axes across diverse models, facilitating interpretability and transferability without model-specific adjustments.

Findings

Behavioral directions align tightly across models in the shared anchor coordinate space.

Transfer of directions achieves high accuracy in downstream tasks, e.g., 0.83 ten-way detection accuracy.

Small anchor pools and a few source models suffice for effective transfer.

Abstract

Large language models from different families use different hidden dimensions, tokenizers, and training procedures, making behavioral directions difficult to compare or transfer across models. We introduce an anchor-projection framework that maps hidden representations from each model into a shared anchor coordinate space (ACS). Behavioral directions extracted from source models are projected into ACS and averaged into a canonical direction. For a new model, the canonical direction is reconstructed into its native hidden space using only anchor activations, without fine-tuning or target-specific direction extraction. We evaluate five instruction-tuned model families and ten behavioral axes. We find that same-axis directions align tightly across the Llama-Qwen-Mistral-Phi (LQMP) cluster in ACS. This shared structure transfers to downstream tasks. For the aligned LQMP cluster, held-out targets achieve (0.83) ten-way detection accuracy and (0.95) mean binary AUROC, while canonical steering induces refusal-rate shifts of up to +0.46% under distribution shift. Sensitivity analyses show that two source models and small anchor pools already suffice to approximate transferable directions. Overall, ACS provides a novel perspective on cross-family interpretability, revealing that representation-level transfer remains robust across model families.