SafeNeuron: Neuron-Level Safety Alignment for Large Language Models

TL;DR

SafeNeuron introduces a neuron-level safety alignment method for large language models that enhances robustness by redistributing safety representations, making safety behaviors more stable and less vulnerable to attacks.

Contribution

It is the first to perform neuron-level safety alignment by identifying and freezing safety-related neurons, improving robustness and interpretability of safety mechanisms.

Findings

Significantly improves robustness against neuron pruning attacks.

Reduces risk of models being repurposed for harmful use.

Preserves general capabilities of the models.

Abstract

Large language models (LLMs) and multimodal LLMs are typically safety-aligned before release to prevent harmful content generation. However, recent studies show that safety behaviors are concentrated in a small subset of parameters, making alignment brittle and easily bypassed through neuron-level attacks. Moreover, most existing alignment methods operate at the behavioral level, offering limited control over the model's internal safety mechanisms. In this work, we propose SafeNeuron, a neuron-level safety alignment framework that improves robustness by redistributing safety representations across the network. SafeNeuron first identifies safety-related neurons, then freezes these neurons during preference optimization to prevent reliance on sparse safety pathways and force the model to construct redundant safety representations. Extensive experiments across models and modalities demonstrate that SafeNeuron significantly improves robustness against neuron pruning attacks, reduces the risk of open-source models being repurposed as red-team generators, and preserves general capabilities. Furthermore, our layer-wise analysis reveals that safety behaviors are governed by stable and shared internal representations. Overall, SafeNeuron provides an interpretable and robust perspective for model alignment.