Make the Unhearable Visible: Exploring Visualization for Musical Instrument Practice

TL;DR

This paper investigates how interactive visualization can support musicians by making unseen patterns in their playing visible, aiding practice and reflection through a series of design explorations and user feedback.

Contribution

It presents a comprehensive design exploration of 33 visualization concepts tailored for musical instrument practice, grounded in musician experience and user feedback.

Findings

Identified key design considerations for visual instrument education.

Demonstrated the potential of visualization to support practice across diverse musical needs.

Provided example visualizations illustrating how unhearable aspects can be made visible.

Abstract

We explore the potential of visualization to support musicians in instrument practice through real-time feedback and reflection on their playing. Musicians often struggle to observe the patterns in their playing and interpret them with respect to their goals. Our premise is that these patterns can be made visible with interactive visualization: we can make the unhearable visible. However, understanding the design of such visualizations is challenging: the diversity of needs, including different instruments, skills, musical attributes, and genres, means that any single use case is unlikely to illustrate the broad potential and opportunities. To address this challenge, we conducted a design exploration study where we created and iterated on 33 designs, each focusing on a subset of needs, for example, only one musical skill. Our designs are grounded in our own experience as musicians and the ideas and feedback of 18 musicians with various musical backgrounds and we evaluated them with 13 music learners and teachers. This paper presents the results of our exploration, focusing on a few example designs as instances of possible instrument practice visualizations. From our work, we draw design considerations that contribute to future research and products for visual instrument education.