The Cost of Thinking: Increased Jailbreak Risk in Large Language Models

TL;DR

This paper reveals that LLMs in thinking mode are more vulnerable to jailbreak attacks, and proposes a safe thinking intervention method to mitigate this risk, improving model safety.

Contribution

It uncovers the increased jailbreak risk in LLMs' thinking mode and introduces a safe thinking intervention technique to enhance safety.

Findings

Thinking mode increases jailbreak success rates

Safe thinking intervention reduces attack success

Long and harmful questions are more vulnerable

Abstract

Thinking mode has always been regarded as one of the most valuable modes in LLMs. However, we uncover a surprising and previously overlooked phenomenon: LLMs with thinking mode are more easily broken by Jailbreak attack. We evaluate 9 LLMs on AdvBench and HarmBench and find that the success rate of attacking thinking mode in LLMs is almost higher than that of non-thinking mode. Through large numbers of sample studies, it is found that for educational purposes and excessively long thinking lengths are the characteristics of successfully attacked data, and LLMs also give harmful answers when they mostly know that the questions are harmful. In order to alleviate the above problems, this paper proposes a method of safe thinking intervention for LLMs, which explicitly guides the internal thinking processes of LLMs by adding "specific thinking tokens" of LLMs to the prompt. The results demonstrate that the safe thinking intervention can significantly reduce the attack success rate of LLMs with thinking mode.