Understanding and Mitigating Over-refusal for Large Language Models via Safety Representation

TL;DR

This paper analyzes the causes of over-refusal in large language models and proposes MOSR, a method that improves safety without excessive rejection by intervening in safety representations.

Contribution

The paper introduces MOSR, a novel approach that mitigates over-refusal in LLMs by using representation-based techniques and context-aware augmentation.

Findings

MOSR reduces over-refusal more effectively than existing methods.

The approach maintains safety while improving usability.

Representation analysis reveals over-refusal samples lie at safety boundary.

Abstract

Large language models demonstrate powerful capabilities across various natural language processing tasks, yet they also harbor safety vulnerabilities. To enhance LLM safety, various jailbreak defense methods have been proposed to guard against harmful outputs. However, improvements in model safety often come at the cost of severe over-refusal, failing to strike a good balance between safety and usability. In this paper, we first analyze the causes of over-refusal from a representation perspective, revealing that over-refusal samples reside at the boundary between benign and malicious samples. Based on this, we propose MOSR, designed to mitigate over-refusal by intervening the safety representation of LLMs. MOSR incorporates two novel components: (1) Overlap-Aware Loss Weighting, which determines the erasure weight for malicious samples by quantifying their similarity to pseudo-malicious samples in the representation space, and (2) Context-Aware Augmentation, which supplements the necessary context for rejection decisions by adding harmful prefixes before rejection responses. Experiments demonstrate that our method outperforms existing approaches in mitigating over-refusal while largely maintaining safety. Overall, we advocate that future defense methods should strike a better balance between safety and over-refusal.