Mind the beat: detecting audio onsets from EEG recordings of music listening

TL;DR

This paper introduces a deep learning method to detect music onsets from EEG recordings, demonstrating that RNNs outperform traditional spectral-flux methods and establishing a new benchmark for this emerging research area.

Contribution

It is the first to apply neural networks to predict audio onsets from EEG data during music listening, showing the effectiveness of RNNs over FCNs and spectral methods.

Findings

RNN outperforms FCN and spectral-flux methods in onset detection.

The approach generalizes better to unseen data.

Provides initial benchmark results for future research.

Abstract

We propose a deep learning approach to predicting audio event onsets in electroencephalogram (EEG) recorded from users as they listen to music. We use a publicly available dataset containing ten contemporary songs and concurrently recorded EEG. We generate a sequence of onset labels for the songs in our dataset and trained neural networks (a fully connected network (FCN) and a recurrent neural network (RNN)) to parse one second windows of input EEG to predict one second windows of onsets in the audio. We compare our RNN network to both the standard spectral-flux based novelty function and the FCN. We find that our RNN was able to produce results that reflected its ability to generalize better than the other methods. Since there are no pre-existing works on this topic, the numbers presented in this paper may serve as useful benchmarks for future approaches to this research problem.