DeformTune: A Deformable XAI Music Prototype for Non-Musicians

TL;DR

DeformTune is an innovative AI music prototype that uses a tactile deformable interface to make AI-assisted music creation more intuitive and accessible for non-musicians, emphasizing explainability and embodied interaction.

Contribution

The paper introduces DeformTune, combining a deformable tactile interface with MeasureVAE to improve AI music interaction for non-experts, with initial user insights.

Findings

Participants faced unclear control mappings

Limited expressive range was observed

Guidance improved user experience

Abstract

Many existing AI music generation tools rely on text prompts, complex interfaces, or instrument-like controls, which may require musical or technical knowledge that non-musicians do not possess. This paper introduces DeformTune, a prototype system that combines a tactile deformable interface with the MeasureVAE model to explore more intuitive, embodied, and explainable AI interaction. We conducted a preliminary study with 11 adult participants without formal musical training to investigate their experience with AI-assisted music creation. Thematic analysis of their feedback revealed recurring challenge--including unclear control mappings, limited expressive range, and the need for guidance throughout use. We discuss several design opportunities for enhancing explainability of AI, including multimodal feedback and progressive interaction support. These findings contribute early insights toward making AI music systems more explainable and empowering for novice users.