Relating Human Perception of Musicality to Prediction in a Predictive Coding Model

TL;DR

This study demonstrates that a predictive coding neural network, trained on music, correlates with human perception of musicality, showing greater prediction errors for less musical sequences and highlighting features influencing musical perception.

Contribution

The paper adapts a predictive coding model from vision to audition, showing its effectiveness in modeling human musical perception and identifying features affecting musicality judgments.

Findings

Prediction errors are higher for less musical sequences.

Prediction error depends on pitch interval and temporal context.

Model trained on music correlates with human judgments of musicality.

Abstract

We explore the use of a neural network inspired by predictive coding for modeling human music perception. This network was developed based on the computational neuroscience theory of recurrent interactions in the hierarchical visual cortex. When trained with video data using self-supervised learning, the model manifests behaviors consistent with human visual illusions. Here, we adapt this network to model the hierarchical auditory system and investigate whether it will make similar choices to humans regarding the musicality of a set of random pitch sequences. When the model is trained with a large corpus of instrumental classical music and popular melodies rendered as mel spectrograms, it exhibits greater prediction errors for random pitch sequences that are rated less musical by human subjects. We found that the prediction error depends on the amount of information regarding the subsequent note, the pitch interval, and the temporal context. Our findings suggest that predictability is correlated with human perception of musicality and that a predictive coding neural network trained on music can be used to characterize the features and motifs contributing to human perception of music.