Learning Features of Music from Scratch

TL;DR

This paper presents MusicNet, a large-scale classical music dataset with annotations, and benchmarks various machine learning models for note prediction, demonstrating that end-to-end models learn frequency-specific audio features.

Contribution

Introduction of MusicNet, a comprehensive annotated music dataset, and evaluation of multiple machine learning architectures for music note prediction.

Findings

End-to-end models learn frequency selective filters.

Spectrogram-based models serve as baselines.

End-to-end neural nets outperform traditional methods.

Abstract

This paper introduces a new large-scale music dataset, MusicNet, to serve as a source of supervision and evaluation of machine learning methods for music research. MusicNet consists of hundreds of freely-licensed classical music recordings by 10 composers, written for 11 instruments, together with instrument/note annotations resulting in over 1 million temporal labels on 34 hours of chamber music performances under various studio and microphone conditions. The paper defines a multi-label classification task to predict notes in musical recordings, along with an evaluation protocol, and benchmarks several machine learning architectures for this task: i) learning from spectrogram features; ii) end-to-end learning with a neural net; iii) end-to-end learning with a convolutional neural net. These experiments show that end-to-end models trained for note prediction learn frequency selective filters as a low-level representation of audio.