HumorReject: Decoupling LLM Safety from Refusal Prefix via A Little Humor

TL;DR

HumorReject introduces a novel approach to LLM safety by using humor as an indirect refusal method, reducing reliance on explicit refusal prefixes and enhancing robustness against attacks.

Contribution

It presents a data-driven method that decouples safety from refusal prefixes using humor, addressing over-defense issues and improving robustness.

Findings

Effective defusing of harmful requests with humor

Superior robustness against attack vectors

Highlights importance of training data design

Abstract

Large Language Models (LLMs) commonly rely on explicit refusal prefixes for safety, making them vulnerable to prefix injection attacks. We introduce HumorReject, a novel data-driven approach that reimagines LLM safety by decoupling it from refusal prefixes through humor as an indirect refusal strategy. Rather than explicitly rejecting harmful instructions, HumorReject responds with contextually appropriate humor that naturally defuses potentially dangerous requests. Our approach effectively addresses common "over-defense" issues while demonstrating superior robustness against various attack vectors. Our findings suggest that improvements in training data design can be as important as the alignment algorithm itself in achieving effective LLM safety. The code and dataset are available at https://github.com/wooozihui/HumorReject.