Tactile Melodies: A Desk-Mounted Haptics for Perceiving Musical Experiences

TL;DR

This paper presents a novel haptic interface that translates MIDI data into tactile sensations on the palm, enabling users to perceive and interpret musical melodies and rhythms through touch, with promising results from a pilot user study.

Contribution

It introduces a real-time music-to-haptic translation system with a strategic mapping approach and evaluates user perception and identification accuracy in musical haptic experiences.

Findings

Users could identify melodies and rhythms through haptic feedback.

Participants with musical background performed better in identifying music.

The system shows potential for enhancing musical experience via tactile interfaces.

Abstract

This paper introduces a novel interface for experiencing music through haptic impulses to the palm of the hand. It presents a practical implementation of the system exploring the realm of musical haptics through the translation of MIDI data from a Digital Audio Workstation (DAW) into haptic sensations, from a set of haptic actuators, in real-time. It also includes a suitable music-to-haptic mapping strategy to translate notes from musical instruments to haptic feedback. The haptic actuators, placed strategically on the palmar surface of the hand allowed users to perceive music and were able to identify melody and rhythm of different musical compositions. A pilot user study conducted intended to assess the accuracy of the interface by testing the participants to select the correct audio presentation from the haptic presentation of the same musical composition. It presents a comparative study, differentiating between those with prior musical background and those without, in identifying the correct audio counterpart solely through haptic inputs. This pilot study delves into how users perceive and interpret haptic feedback within the context of musical compositions. The study showed promising results in enriching our understanding of user responses to haptic feedback in musical scenarios and exploring the intricacies of user experience with the system and its impact on musical interpretation.