Piano Learning and Improvisation through Adaptive Visualisation and Digital Augmentation

TL;DR

This paper proposes an adaptive visualisation system for piano learning that personalizes support based on learner proficiency and encourages improvisation through expert annotations, aiming to enhance sight-reading and improvisational skills.

Contribution

It introduces a novel adaptive visualisation framework for piano learning that integrates learner proficiency and expert guidance to improve practice and improvisation.

Findings

Supports personalized piano practice through adaptive visualisations

Enhances improvisation with expert annotations in an interactive space

Demonstrates improved learning outcomes via user studies

Abstract

The task of learning the piano has been a centuries-old challenge for novices, experts and technologists. Several innovations have been introduced to support proper posture, movement, and motivation, while sight-reading and improvisation remain the least-explored areas. In this PhD, we address this gap by redesigning the piano augmentation as an interactive and adaptive space. Specifically, we will explore how to support learners with adaptive visualisations through a two-pronged approach: (1) by designing adaptive visualisations based on the proficiency of the learner to support regular piano playing and (2) by assisting them with expert annotations projected on the piano to encourage improvisation. To this end, we will build a model to understand the complexities of learners' spatiotemporal data and use these to support learning. We will then evaluate our approach through user studies enabling practice and improvisation. Our work contributes to how adaptive visualisations can push music instrument learning and support multi-target selection tasks in immersive spaces.