FRACTURED-SORRY-Bench: Framework for Revealing Attacks in Conversational Turns Undermining Refusal Efficacy and Defenses over SORRY-Bench (Automated Multi-shot Jailbreaks)

TL;DR

This paper presents FRACTURED-SORRY-Bench, a framework for testing LLM safety against multi-turn conversational attacks, revealing vulnerabilities in current defenses through a novel adversarial prompt generation method.

Contribution

It introduces a new framework and method for generating adversarial prompts that expose weaknesses in LLM safety measures against multi-turn attacks.

Findings

Attack success rates increased by up to 46.22%

The method effectively challenges existing safety defenses

Highlights need for more robust LLM safety mechanisms

Abstract

This paper introduces FRACTURED-SORRY-Bench, a framework for evaluating the safety of Large Language Models (LLMs) against multi-turn conversational attacks. Building upon the SORRY-Bench dataset, we propose a simple yet effective method for generating adversarial prompts by breaking down harmful queries into seemingly innocuous sub-questions. Our approach achieves a maximum increase of +46.22\% in Attack Success Rates (ASRs) across GPT-4, GPT-4o, GPT-4o-mini, and GPT-3.5-Turbo models compared to baseline methods. We demonstrate that this technique poses a challenge to current LLM safety measures and highlights the need for more robust defenses against subtle, multi-turn attacks.