Obscure but Effective: Classical Chinese Jailbreak Prompt Optimization via Bio-Inspired Search

TL;DR

This paper introduces CC-BOS, a bio-inspired search framework that automatically generates classical Chinese prompts to effectively bypass LLM safety constraints, revealing vulnerabilities and improving attack success rates.

Contribution

It presents a novel multi-dimensional fruit fly optimization approach for generating classical Chinese jailbreak prompts, enhancing black-box attack efficiency and effectiveness.

Findings

CC-BOS outperforms existing jailbreak methods in success rate

The framework effectively explores the prompt search space

Classical Chinese prompts exploit LLM vulnerabilities

Abstract

As Large Language Models (LLMs) are increasingly used, their security risks have drawn increasing attention. Existing research reveals that LLMs are highly susceptible to jailbreak attacks, with effectiveness varying across language contexts. This paper investigates the role of classical Chinese in jailbreak attacks. Owing to its conciseness and obscurity, classical Chinese can partially bypass existing safety constraints, exposing notable vulnerabilities in LLMs. Based on this observation, this paper proposes a framework, CC-BOS, for the automatic generation of classical Chinese adversarial prompts based on multi-dimensional fruit fly optimization, facilitating efficient and automated jailbreak attacks in black-box settings. Prompts are encoded into eight policy dimensions-covering role, behavior, mechanism, metaphor, expression, knowledge, trigger pattern and context; and iteratively refined via smell search, visual search, and cauchy mutation. This design enables efficient exploration of the search space, thereby enhancing the effectiveness of black-box jailbreak attacks. To enhance readability and evaluation accuracy, we further design a classical Chinese to English translation module. Extensive experiments demonstrate that effectiveness of the proposed CC-BOS, consistently outperforming state-of-the-art jailbreak attack methods.