A Sensitivity Analysis of Multi-Event Audio Grounding in Audio LLMs

TL;DR

This study evaluates how increasing complexity in audio scenes affects the accuracy and reliability of state-of-the-art Audio LLMs in event grounding, revealing significant challenges and areas for improvement.

Contribution

It provides the first large-scale, systematic analysis of multi-event audio grounding in Audio LLMs, highlighting their limitations and the impact of prompt design on performance.

Findings

Increasing event count reduces true-positive rate.

More events increase false-positive rate.

Prompt design significantly affects model trade-offs.

Abstract

Audio LLMs have shown a strong ability to understand audio samples, yet their reliability in complex acoustic scenes remains under-explored. Unlike prior work limited to small scale or less controlled query construction, we present a large-scale evaluation of event grounding and false alarms as auditory scene complexity increases. Using 71K AudioCapsV2 clips, we extract normalized (source, attribute) events and build two query types: present-event queries for ground-truth detection and absent-event queries to probe hallucinations, using similarity-filtered negative sampling in an audio-aligned text embedding space. We evaluate four SOTA Audio LLMs with 12 prompt variants over 500K yes/no queries per model. Across models, increasing event count consistently lowers true-positive rate and raises false-positive rate, while prompts induce a strong trade-off between the two. Our confidence analysis shows that models become more uncertain on multi-event audio, revealing room for improvement.