Play Me Something Icy: Practical Challenges, Explainability and the Semantic Gap in Generative AI Music

TL;DR

This paper critically examines the challenges, explainability issues, and semantic gaps in text-to-audio and text-to-music generative AI tools from the perspective of experimental musicians and researchers.

Contribution

It highlights key challenges like semantic gaps and explainability in generative AI music, providing insights and suggestions for future improvements.

Findings

Semantic gap in describing music with text-based prompts

Trade-off between explainability and usability in AI tools

Need for better user control and input mechanisms

Abstract

This pictorial aims to critically consider the nature of text-to-audio and text-to-music generative tools in the context of explainable AI. As a group of experimental musicians and researchers, we are enthusiastic about the creative potential of these tools and have sought to understand and evaluate them from perspectives of prompt creation, control, usability, understandability, explainability of the AI process, and overall aesthetic effectiveness of the results. One of the challenges we have identified that is not explicitly addressed by these tools is the inherent semantic gap in using text-based tools to describe something as abstract as music. Other gaps include explainability vs. useability, and user control and input vs. the human creative process. The aim of this pictorial is to raise questions for discussion and make a few general suggestions on the kinds of improvements we would like to see in generative AI music tools.