Look Before You Leap: Enhancing Attention and Vigilance Regarding Harmful Content with GuidelineLLM

TL;DR

This paper introduces GuidelineLLM, a new method that improves LLM safety by identifying harmful queries and providing guidelines before response, reducing jailbreak success rates without extra fine-tuning.

Contribution

The paper presents GuidelineLLM, a novel framework that enhances LLM safety by pre-emptively recognizing risks and guiding responses, avoiding additional fine-tuning of the LLMs.

Findings

Significantly reduces attack success rate by 34.17% on average.

Maintains usefulness of LLMs for benign queries.

Does not require additional safety fine-tuning of LLMs.

Abstract

Despite being empowered with alignment mechanisms, large language models (LLMs) are increasingly vulnerable to emerging jailbreak attacks that can compromise their alignment mechanisms. This vulnerability poses significant risks to real-world applications. Existing work faces challenges in both training efficiency and generalization capabilities (i.e., Reinforcement Learning from Human Feedback and Red-Teaming). Developing effective strategies to enable LLMs to resist continuously evolving jailbreak attempts represents a significant challenge. To address this challenge, we propose a novel defensive paradigm called GuidelineLLM, which assists LLMs in recognizing queries that may have harmful content. Before LLMs respond to a query, GuidelineLLM first identifies potential risks associated with the query, summarizes these risks into guideline suggestions, and then feeds these guidelines to the responding LLMs. Importantly, our approach eliminates the necessity for additional safety fine-tuning of the LLMs themselves; only the GuidelineLLM requires fine-tuning. This characteristic enhances the general applicability of GuidelineLLM across various LLMs. Experimental results demonstrate that GuidelineLLM can significantly reduce the attack success rate (ASR) against LLM (an average reduction of 34.17\% ASR) while maintaining the usefulness of LLM in handling benign queries. The code is available at https://github.com/sqzhang-lazy/GuidelineLLM.