Evaluating Compositional Structure in Audio Representations

TL;DR

This paper introduces a new benchmark to evaluate how well audio representations capture compositional structures, inspired by similar concepts in vision and language, using synthetic datasets and two novel tasks.

Contribution

It presents the first benchmark for assessing compositionality in audio embeddings, with tasks and datasets designed to systematically evaluate this property.

Findings

Introduces A-COAT and A-TRE tasks for evaluating compositionality.

Provides synthetic datasets with controlled acoustic attribute variations.

Establishes a baseline for future research in audio representation evaluation.

Abstract

We propose a benchmark for evaluating compositionality in audio representations. Audio compositionality refers to representing sound scenes in terms of constituent sources and attributes, and combining them systematically. While central to auditory perception, this property is largely absent from current evaluation protocols. Our framework adapts ideas from vision and language to audio through two tasks: A-COAT, which tests consistency under additive transformations, and A-TRE, which probes reconstructibility from attribute-level primitives. Both tasks are supported by large synthetic datasets with controlled variation in acoustic attributes, providing the first benchmark of compositional structure in audio embeddings.