Affective Music Information Retrieval

TL;DR

This paper introduces a novel probabilistic generative model called Acoustic Emotion Gaussians (AEG) for modeling music emotions in the valence-arousal space, improving emotion recognition and retrieval in music information retrieval.

Contribution

The paper presents a new Gaussian mixture model approach that captures emotion perception subjectivity and supports online learning for personalized and general emotion-based music applications.

Findings

AEG effectively models emotion perception in music.

The model performs well on large-scale emotion-annotated datasets.

AEG supports both user-independent and user-dependent emotion recognition.

Abstract

Much of the appeal of music lies in its power to convey emotions/moods and to evoke them in listeners. In consequence, the past decade witnessed a growing interest in modeling emotions from musical signals in the music information retrieval (MIR) community. In this article, we present a novel generative approach to music emotion modeling, with a specific focus on the valence-arousal (VA) dimension model of emotion. The presented generative model, called \emph{acoustic emotion Gaussians} (AEG), better accounts for the subjectivity of emotion perception by the use of probability distributions. Specifically, it learns from the emotion annotations of multiple subjects a Gaussian mixture model in the VA space with prior constraints on the corresponding acoustic features of the training music pieces. Such a computational framework is technically sound, capable of learning in an online fashion, and thus applicable to a variety of applications, including user-independent (general) and user-dependent (personalized) emotion recognition and emotion-based music retrieval. We report evaluations of the aforementioned applications of AEG on a larger-scale emotion-annotated corpora, AMG1608, to demonstrate the effectiveness of AEG and to showcase how evaluations are conducted for research on emotion-based MIR. Directions of future work are also discussed.