Zero-shot Learning for Audio-based Music Classification and Tagging

TL;DR

This paper explores zero-shot learning for music classification and tagging, enabling recognition of unseen labels by leveraging semantic label information and proposing new evaluation schemes for multi-label scenarios.

Contribution

It introduces two setups of semantic side information for zero-shot music classification and tagging, along with a novel data split scheme for multi-label zero-shot learning evaluation.

Findings

Zero-shot learning improves recognition of unseen music labels.

Semantic side information enhances multi-label classification performance.

Proposed evaluation schemes facilitate future research in zero-shot music tasks.

Abstract

Audio-based music classification and tagging is typically based on categorical supervised learning with a fixed set of labels. This intrinsically cannot handle unseen labels such as newly added music genres or semantic words that users arbitrarily choose for music retrieval. Zero-shot learning can address this problem by leveraging an additional semantic space of labels where side information about the labels is used to unveil the relationship between each other. In this work, we investigate the zero-shot learning in the music domain and organize two different setups of side information. One is using human-labeled attribute information based on Free Music Archive and OpenMIC-2018 datasets. The other is using general word semantic information based on Million Song Dataset and Last.fm tag annotations. Considering a music track is usually multi-labeled in music classification and tagging datasets, we also propose a data split scheme and associated evaluation settings for the multi-label zero-shot learning. Finally, we report experimental results and discuss the effectiveness and new possibilities of zero-shot learning in the music domain.