Joint Transcription of Acoustic Guitar Strumming Directions and Chords

TL;DR

This paper presents a deep learning approach for automatic transcription of guitar strumming, including directions and chords, using a novel dataset collected with a smartwatch sensor and synthetic data, achieving improved accuracy over baselines.

Contribution

Introduces a multimodal deep learning model and a new dataset for guitar strumming transcription, enhancing accuracy in detecting strumming directions and chords from audio.

Findings

Significant improvement over baseline detection methods.

Hybrid dataset approach yields highest accuracy.

Deep learning enables robust rhythm guitar analysis.

Abstract

Automatic transcription of guitar strumming is an underrepresented and challenging task in Music Information Retrieval (MIR), particularly for extracting both strumming directions and chord progressions from audio signals. While existing methods show promise, their effectiveness is often hindered by limited datasets. In this work, we extend a multimodal approach to guitar strumming transcription by introducing a novel dataset and a deep learning-based transcription model. We collect 90 min of real-world guitar recordings using an ESP32 smartwatch motion sensor and a structured recording protocol, complemented by a synthetic dataset of 4h of labeled strumming audio. A Convolutional Recurrent Neural Network (CRNN) model is trained to detect strumming events, classify their direction, and identify the corresponding chords using only microphone audio. Our evaluation demonstrates significant improvements over baseline onset detection algorithms, with a hybrid method combining synthetic and real-world data achieving the highest accuracy for both strumming action detection and chord classification. These results highlight the potential of deep learning for robust guitar strumming transcription and open new avenues for automatic rhythm guitar analysis.