RASA: Routing-Aware Safety Alignment for Mixture-of-Experts Models

TL;DR

RASA is a novel routing-aware safety alignment framework for Mixture-of-Experts models that repairs safety-critical experts selectively, achieving high robustness against jailbreak attacks while maintaining overall performance.

Contribution

The paper introduces RASA, a targeted expert repair method that improves safety alignment in MoE models by addressing routing-based bypasses, outperforming prior global fine-tuning approaches.

Findings

RASA achieves near-perfect robustness against jailbreak attacks.

RASA maintains strong performance on benchmarks like MMLU, GSM8K, and TruthfulQA.

Targeted expert repair outperforms global parameter updates for safety alignment.

Abstract

Mixture-of-Experts (MoE) language models introduce unique challenges for safety alignment due to their sparse routing mechanisms, which can enable degenerate optimization behaviors under standard full-parameter fine-tuning. In our preliminary experiments, we observe that naively applying full-parameter safety fine-tuning to MoE models can reduce attack success rates through routing or expert dominance effects, rather than by directly repairing Safety-Critical Experts. To address this challenge, we propose RASA, a routing-aware expert-level alignment framework that explicitly repairs Safety-Critical Experts while preventing routing-based bypasses. RASA identifies experts disproportionately activated by successful jailbreaks, selectively fine-tunes only these experts under fixed routing, and subsequently enforces routing consistency with safety-aligned contexts. Across two representative MoE architectures and a diverse set of jailbreak attacks, RASA achieves near-perfect robustness, strong cross-attack generalization, and substantially reduced over-refusal, while preserving general capabilities on benchmarks such as MMLU, GSM8K, and TruthfulQA. Our results suggest that robust MoE safety alignment benefits from targeted expert repair rather than global parameter updates, offering a practical and architecture-preserving alternative to prior approaches.