Uncertainty Estimation in the Real World: A Study on Music Emotion Recognition

TL;DR

This paper investigates methods for estimating both the central tendency and uncertainty of subjective emotional responses to music, revealing challenges in accurately modeling response variability with current techniques.

Contribution

It introduces and evaluates approaches for uncertainty estimation in music emotion recognition, highlighting the difficulty of modeling subjective response variability.

Findings

Modeling central tendencies is feasible.

Uncertainty estimation remains challenging with current methods.

Empirical response variation estimates do not fully improve uncertainty modeling.

Abstract

Any data annotation for subjective tasks shows potential variations between individuals. This is particularly true for annotations of emotional responses to musical stimuli. While older approaches to music emotion recognition systems frequently addressed this uncertainty problem through probabilistic modeling, modern systems based on neural networks tend to ignore the variability and focus only on predicting central tendencies of human subjective responses. In this work, we explore several methods for estimating not only the central tendencies of the subjective responses to a musical stimulus, but also for estimating the uncertainty associated with these responses. In particular, we investigate probabilistic loss functions and inference-time random sampling. Experimental results indicate that while the modeling of the central tendencies is achievable, modeling of the uncertainty in subjective responses proves significantly more challenging with currently available approaches even when empirical estimates of variations in the responses are available.