Understanding Refusal in Language Models with Sparse Autoencoders

TL;DR

This paper investigates the internal mechanisms of refusal behavior in instruction-tuned language models using sparse autoencoders, revealing causal features and improving understanding of safety behaviors.

Contribution

It introduces a mechanistic approach with sparse autoencoders to identify and intervene on latent features mediating refusals in language models, advancing interpretability.

Findings

Identified latent features causally linked to refusal behavior

Validated intervention effects on refusal across harmful datasets

Enhanced generalization of refusal features for adversarial robustness

Abstract

Refusal is a key safety behavior in aligned language models, yet the internal mechanisms driving refusals remain opaque. In this work, we conduct a mechanistic study of refusal in instruction-tuned LLMs using sparse autoencoders to identify latent features that causally mediate refusal behaviors. We apply our method to two open-source chat models and intervene on refusal-related features to assess their influence on generation, validating their behavioral impact across multiple harmful datasets. This enables a fine-grained inspection of how refusal manifests at the activation level and addresses key research questions such as investigating upstream-downstream latent relationship and understanding the mechanisms of adversarial jailbreaking techniques. We also establish the usefulness of refusal features in enhancing generalization for linear probes to out-of-distribution adversarial samples in classification tasks. We open source our code in https://github.com/wj210/refusal_sae.