AudioGuard: Toward Comprehensive Audio Safety Protection Across Diverse Threat Models

TL;DR

AudioGuard introduces a comprehensive audio safety framework with a new benchmark, addressing diverse threat models and improving detection accuracy with lower latency.

Contribution

The paper presents AudioSafetyBench, the first policy-based audio safety benchmark, and proposes AudioGuard, a unified guardrail for waveform and semantic protection.

Findings

AudioGuard outperforms strong baselines in accuracy.

AudioSafetyBench covers multiple languages and threat types.

AudioGuard achieves lower latency in detection.

Abstract

Audio has rapidly become a primary interface for foundation models, powering real-time voice assistants. Ensuring safety in audio systems is inherently more complex than just "unsafe text spoken aloud": real-world risks can hinge on audio-native harmful sound events, speaker attributes (e.g., child voice), impersonation/voice-cloning misuse, and voice-content compositional harms, such as child voice plus sexual content. The nature of audio makes it challenging to develop comprehensive benchmarks or guardrails against this unique risk landscape. To close this gap, we conduct large-scale red teaming on audio systems, systematically uncover vulnerabilities in audio, and develop a comprehensive, policy-grounded audio risk taxonomy and AudioSafetyBench, the first policy-based audio safety benchmark across diverse threat models. AudioSafetyBench supports diverse languages, suspicious voices (e.g., celebrity/impersonation and child voice), risky voice-content combinations, and non-speech sound events. To defend against these threats, we propose AudioGuard, a unified guardrail consisting of 1) SoundGuard for waveform-level audio-native detection and 2) ContentGuard for policy-grounded semantic protection. Extensive experiments on AudioSafetyBench and four complementary benchmarks show that AudioGuard consistently improves guardrail accuracy over strong audio-LLM-based baselines with substantially lower latency.