Embedding-based Music Emotion Recognition Using Composite Loss

TL;DR

This paper introduces an embedding-based music emotion recognition method that captures both broad and fine-grained emotional variations by using a composite loss to optimize embeddings, demonstrating improved robustness and accuracy.

Contribution

It proposes a novel embedding approach with a composite loss function that considers both correlation and probabilistic similarity for more nuanced emotion recognition.

Findings

Effective on two benchmark datasets

Robust bidirectional emotion recognition achieved

Improved discrimination within emotional categories

Abstract

Most music emotion recognition approaches perform classification or regression that estimates a general emotional category from a distribution of music samples, but without considering emotional variations (e.g., happiness can be further categorised into much, moderate or little happiness). We propose an embedding-based music emotion recognition approach that associates music samples with emotions in a common embedding space by considering both general emotional categories and fine-grained discrimination within each category. Since the association of music samples with emotions is uncertain due to subjective human perceptions, we compute composite loss-based embeddings obtained to maximise two statistical characteristics, one being the correlation between music samples and emotions based on canonical correlation analysis, and the other being a probabilistic similarity between a music sample and an emotion with KL-divergence. The experiments on two benchmark datasets demonstrate the effectiveness of our embedding-based approach, the composite loss and learned acoustic features. In addition, detailed analysis shows that our approach can accomplish robust bidirectional music emotion recognition that not only identifies music samples matching with a specific emotion but also detects emotions expressed in a certain music sample.