Expressive Range Characterization of Open Text-to-Audio Models

TL;DR

This paper adapts expressive range analysis to evaluate the variability and fidelity of open text-to-audio models by analyzing their outputs across key acoustic dimensions for standardized prompts.

Contribution

It introduces a framework for applying expressive range analysis to text-to-audio models, enabling systematic evaluation of their output diversity and quality.

Findings

Analyzed model outputs along acoustic dimensions like pitch and timbre.

Demonstrated variability in generated audio for fixed prompts.

Provided a new evaluation framework for generative audio models.

Abstract

Text-to-audio models are a type of generative model that produces audio output in response to a given textual prompt. Although level generators and the properties of the functional content that they create (e.g., playability) dominate most discourse in procedurally generated content (PCG), games that emotionally resonate with players tend to weave together a range of creative and multimodal content (e.g., music, sounds, visuals, narrative tone), and multimodal models have begun seeing at least experimental use for this purpose. However, it remains unclear what exactly such models generate, and with what degree of variability and fidelity: audio is an extremely broad class of output for a generative system to target. Within the PCG community, expressive range analysis (ERA) has been used as a quantitative way to characterize generators' output space, especially for level generators. This paper adapts ERA to text-to-audio models, making the analysis tractable by looking at the expressive range of outputs for specific, fixed prompts. Experiments are conducted by prompting the models with several standardized prompts derived from the Environmental Sound Classification (ESC-50) dataset. The resulting audio is analyzed along key acoustic dimensions (e.g., pitch, loudness, and timbre). More broadly, this paper offers a framework for ERA-based exploratory evaluation of generative audio models.