Automatic Detection and Analysis of Singing Mistakes for Music Pedagogy

TL;DR

This paper presents a machine learning framework for automatic singing mistake detection in music education, including a new dataset, models, and evaluation methods, demonstrating superior performance over rule-based approaches.

Contribution

Introduces a novel deep learning-based system for singing mistake detection with a curated dataset and evaluation methodology, advancing music pedagogy research.

Findings

Deep learning models outperform rule-based methods

New dataset with annotated singing mistakes created

Insights into pedagogical error patterns obtained

Abstract

The advancement of machine learning in audio analysis has opened new possibilities for technology-enhanced music education. This paper introduces a framework for automatic singing mistake detection in the context of music pedagogy, supported by a newly curated dataset. The dataset comprises synchronized teacher learner vocal recordings, with annotations marking different types of mistakes made by learners. Using this dataset, we develop different deep learning models for mistake detection and benchmark them. To compare the efficacy of mistake detection systems, a new evaluation methodology is proposed. Experiments indicate that the proposed learning-based methods are superior to rule-based methods. A systematic study of errors and a cross-teacher study reveal insights into music pedagogy that can be utilised for various music applications. This work sets out new directions of research in music pedagogy. The codes and dataset are publicly available.