AdvWave: Stealthy Adversarial Jailbreak Attack against Large Audio-Language Models

TL;DR

This paper introduces AdvWave, a novel framework for stealthy adversarial attacks on large audio-language models, overcoming technical challenges like gradient shattering and behavioral variability to effectively jailbreak these models.

Contribution

AdvWave is the first comprehensive jailbreak framework for LALMs, featuring a dual-phase optimization, adaptive target search, and classifier-guided naturalistic adversarial audio generation.

Findings

Achieves 40% higher success rate than baseline methods.

Effectively overcomes gradient shattering in LALMs.

Generates perceptually natural adversarial audio.

Abstract

Recent advancements in large audio-language models (LALMs) have enabled speech-based user interactions, significantly enhancing user experience and accelerating the deployment of LALMs in real-world applications. However, ensuring the safety of LALMs is crucial to prevent risky outputs that may raise societal concerns or violate AI regulations. Despite the importance of this issue, research on jailbreaking LALMs remains limited due to their recent emergence and the additional technical challenges they present compared to attacks on DNN-based audio models. Specifically, the audio encoders in LALMs, which involve discretization operations, often lead to gradient shattering, hindering the effectiveness of attacks relying on gradient-based optimizations. The behavioral variability of LALMs further complicates the identification of effective (adversarial) optimization targets. Moreover, enforcing stealthiness constraints on adversarial audio waveforms introduces a reduced, non-convex feasible solution space, further intensifying the challenges of the optimization process. To overcome these challenges, we develop AdvWave, the first jailbreak framework against LALMs. We propose a dual-phase optimization method that addresses gradient shattering, enabling effective end-to-end gradient-based optimization. Additionally, we develop an adaptive adversarial target search algorithm that dynamically adjusts the adversarial optimization target based on the response patterns of LALMs for specific queries. To ensure that adversarial audio remains perceptually natural to human listeners, we design a classifier-guided optimization approach that generates adversarial noise resembling common urban sounds. Extensive evaluations on multiple advanced LALMs demonstrate that AdvWave outperforms baseline methods, achieving a 40% higher average jailbreak attack success rate.