Representations of Sound in Deep Learning of Audio Features from Music

TL;DR

This paper explores how deep convolutional neural networks can learn subtle stylistic clues in music to classify composers, demonstrating high accuracy with various audio representations including spectral and random matrix transforms.

Contribution

It introduces the use of a deep CNN with novel audio representations like RMT for composer classification, showing improved performance over traditional methods.

Findings

RMT representation achieved 84% accuracy in composer classification.

Logarithmic filter bank representation achieved 68% accuracy.

Deep CNN effectively captures stylistic features in music.

Abstract

The work of a single musician, group or composer can vary widely in terms of musical style. Indeed, different stylistic elements, from performance medium and rhythm to harmony and texture, are typically exploited and developed across an artist's lifetime. Yet, there is often a discernable character to the work of, for instance, individual composers at the perceptual level - an experienced listener can often pick up on subtle clues in the music to identify the composer or performer. Here we suggest that a convolutional network may learn these subtle clues or features given an appropriate representation of the music. In this paper, we apply a deep convolutional neural network to a large audio dataset and empirically evaluate its performance on audio classification tasks. Our trained network demonstrates accurate performance on such classification tasks when presented with 5 s examples of music obtained by simple transformations of the raw audio waveform. A particularly interesting example is the spectral representation of music obtained by application of a logarithmically spaced filter bank, mirroring the early stages of auditory signal transduction in mammals. The most successful representation of music to facilitate discrimination was obtained via a random matrix transform (RMT). Networks based on logarithmic filter banks and RMT were able to correctly guess the one composer out of 31 possibilities in 68 and 84 percent of cases respectively.