Defense Against the Dark Prompts: Mitigating Best-of-N Jailbreaking with Prompt Evaluation

TL;DR

The paper introduces DATDP, a prompt evaluation method that effectively detects and blocks jailbreaking attempts in large language models, significantly enhancing AI safety.

Contribution

It presents a novel prompt evaluation approach that explicitly detects jailbreaking prompts, improving safety across various LLMs with minimal added cost.

Findings

DATDP blocked 100% of successful jailbreaks in tested scenarios.

The method is effective even with smaller evaluation models like LLaMa-3-8B.

Adding DATDP significantly increases safety in generative AI systems.

Abstract

Recent work showed Best-of-N (BoN) jailbreaking using repeated use of random augmentations (such as capitalization, punctuation, etc) is effective against all major large language models (LLMs). We have found that $100\%$ of the BoN paper's successful jailbreaks (confidence interval $[99.65\%, 100.00\%]$) and $99.8\%$ of successful jailbreaks in our replication (confidence interval $[99.28\%, 99.98\%]$) were blocked with our Defense Against The Dark Prompts (DATDP) method. The DATDP algorithm works by repeatedly utilizing an evaluation LLM to evaluate a prompt for dangerous or manipulative behaviors--unlike some other approaches, DATDP also explicitly looks for jailbreaking attempts--until a robust safety rating is generated. This success persisted even when utilizing smaller LLMs to power the evaluation (Claude and LLaMa-3-8B-instruct proved almost equally capable). These results show that, though language models are sensitive to seemingly innocuous changes to inputs, they seem also capable of successfully evaluating the dangers of these inputs. Versions of DATDP can therefore be added cheaply to generative AI systems to produce an immediate significant increase in safety.