Designing a Generative AI-Assisted Music Psychotherapy Tool for Deaf and Hard-of-Hearing Individuals

TL;DR

This paper introduces a co-designed AI-assisted music therapy tool for Deaf and Hard-of-Hearing individuals, enabling emotional expression and self-awareness through collaborative songwriting.

Contribution

It presents a novel AI-based music psychotherapy tool specifically designed for DHH individuals, incorporating conversational agents and visual metaphors.

Findings

DHH participants experienced emotional release and self-understanding through AI-assisted songwriting.

The CA's empathy and visual metaphors effectively facilitated musical dialogue for DHH users.

The study demonstrates AI's potential to make music therapy inclusive for DHH populations.

Abstract

Songwriting has long served as a powerful medium for expressing unconscious emotions and fostering self-awareness in psychotherapy. Due to the auditory-centric nature of traditional approaches, Deaf and Hard-of-Hearing (DHH) individuals have often been excluded from music's therapeutic benefits. In response, this study presents a music psychotherapy tool co-designed with therapists, integrating conversational agents (CAs) and music generative AI as symbolic and therapeutic media. Through a usage study with 23 DHH individuals, we found that collaborative song writing with the CA enabled them to experience emotional release, reinterpretation, and deeper self-understanding. In particular, the CA's strategies -- supportive empathy, example response options, and visual-based metaphors -- were found to facilitate musical dialogue effectively for DHH individuals. These findings contribute to inclusive AI design by showing the potential of human-AI collaboration to bridge therapeutic artistic practices.