ORFuzz: Fuzzing the "Other Side" of LLM Safety -- Testing Over-Refusal

TL;DR

ORFuzz is an innovative evolutionary testing framework designed to systematically detect and analyze over-refusal behaviors in large language models, enhancing their reliability and safety.

Contribution

It introduces the first evolutionary testing approach for LLM over-refusal, combining safety-aware seed selection, adaptive mutator optimization, and a human-aligned judge model.

Findings

ORFuzz generates over-refusal instances at over twice the rate of baselines.

It creates ORFuzzSet, a benchmark with 1,855 transferable test cases.

The benchmark achieves a 63.56% average over-refusal rate across 10 LLMs.

Abstract

Large Language Models (LLMs) increasingly exhibit over-refusal - erroneously rejecting benign queries due to overly conservative safety measures - a critical functional flaw that undermines their reliability and usability. Current methods for testing this behavior are demonstrably inadequate, suffering from flawed benchmarks and limited test generation capabilities, as highlighted by our empirical user study. To the best of our knowledge, this paper introduces the first evolutionary testing framework, ORFuzz, for the systematic detection and analysis of LLM over-refusals. ORFuzz uniquely integrates three core components: (1) safety category-aware seed selection for comprehensive test coverage, (2) adaptive mutator optimization using reasoning LLMs to generate effective test cases, and (3) OR-Judge, a human-aligned judge model validated to accurately reflect user perception of toxicity and refusal. Our extensive evaluations demonstrate that ORFuzz generates diverse, validated over-refusal instances at a rate (6.98% average) more than double that of leading baselines, effectively uncovering vulnerabilities. Furthermore, ORFuzz's outputs form the basis of ORFuzzSet, a new benchmark of 1,855 highly transferable test cases that achieves a superior 63.56% average over-refusal rate across 10 diverse LLMs, significantly outperforming existing datasets. ORFuzz and ORFuzzSet provide a robust automated testing framework and a valuable community resource, paving the way for developing more reliable and trustworthy LLM-based software systems.