MelodyVis: Visual Analytics for Melodic Patterns in Sheet Music

TL;DR

MelodyVis is a visual analytics tool that helps musicologists explore and identify melodic patterns in digital sheet music more effectively than manual or fully automated methods.

Contribution

The paper introduces MelodyVis, a novel visual system with interactive operators for melodic pattern analysis, developed in collaboration with musicology experts.

Findings

Participants identified twice as many patterns with operators enabled.

MelodyVis improved pattern interpretation and analysis efficiency.

User study confirmed the system's usefulness and usability.

Abstract

Manual melody detection is a tedious task requiring high expertise level, while automatic detection is often not expressive or powerful enough. Thus, we present MelodyVis, a visual application designed in collaboration with musicology experts to explore melodic patterns in digital sheet music. MelodyVis features five connected views, including a Melody Operator Graph and a Voicing Timeline. The system utilizes eight atomic operators, such as transposition and mirroring, to capture melody repetitions and variations. Users can start their analysis by manually selecting patterns in the sheet view, and then identifying other patterns based on the selected samples through an interactive exploration process. We conducted a user study to investigate the effectiveness and usefulness of our approach and its integrated melodic operators, including usability and mental load questions. We compared the analysis executed by 25 participants with and without the operators. The study results indicate that the participants could identify at least twice as many patterns with activated operators. MelodyVis allows analysts to steer the analysis process and interpret results. Our study also confirms the usefulness of MelodyVis in supporting common analytical tasks in melodic analysis, with participants reporting improved pattern identification and interpretation. Thus, MelodyVis addresses the limitations of fully-automated approaches, enabling music analysts to step into the analysis process and uncover and understand intricate melodic patterns and transformations in sheet music.