Musical instrument sound classification with deep convolutional neural network using feature fusion approach

TL;DR

This paper introduces a deep learning approach for musical instrument classification that fuses features from spectrograms and multiresolution recurrence plots, leveraging CNNs to improve accuracy over traditional methods.

Contribution

The paper proposes a novel feature fusion method combining spectrograms and MRPs with CNNs for enhanced instrument classification performance.

Findings

Fusion of spectrograms and MRPs improves classification accuracy.

The proposed CNN-based system outperforms traditional handcrafted feature methods.

Multiresolution recurrence plots capture phase information not available in spectrograms.

Abstract

A new musical instrument classification method using convolutional neural networks (CNNs) is presented in this paper. Unlike the traditional methods, we investigated a scheme for classifying musical instruments using the learned features from CNNs. To create the learned features from CNNs, we not only used a conventional spectrogram image, but also proposed multiresolution recurrence plots (MRPs) that contain the phase information of a raw input signal. Consequently, we fed the characteristic timbre of the particular instrument into a neural network, which cannot be extracted using a phase-blinded representations such as a spectrogram. By combining our proposed MRPs and spectrogram images with a multi-column network, the performance of our proposed classifier system improves over a system that uses only a spectrogram. Furthermore, the proposed classifier also outperforms the baseline result from traditional handcrafted features and classifiers.