MERGE -- A Bimodal Audio-Lyrics Dataset for Static Music Emotion Recognition

TL;DR

This paper introduces MERGE, a set of three large, quality-controlled bimodal audio-lyrics datasets for music emotion recognition, enabling improved benchmarking and development of multimodal systems.

Contribution

The paper presents three new publicly available bimodal datasets created with a semi-automatic approach, along with baseline experiments and validated data splits for music emotion recognition research.

Findings

Achieved up to 81.74% F1-score in bimodal classification

Validated the datasets' viability through extensive experiments

Provided standardized train-validation-test splits for benchmarking

Abstract

The Music Emotion Recognition (MER) field has seen steady developments in recent years, with contributions from feature engineering, machine learning, and deep learning. The landscape has also shifted from audio-centric systems to bimodal ensembles that combine audio and lyrics. However, a lack of public, sizable and quality-controlled bimodal databases has hampered the development and improvement of bimodal audio-lyrics systems. This article proposes three new audio, lyrics, and bimodal MER research datasets, collectively referred to as MERGE, which were created using a semi-automatic approach. To comprehensively assess the proposed datasets and establish a baseline for benchmarking, we conducted several experiments for each modality, using feature engineering, machine learning, and deep learning methodologies. Additionally, we propose and validate fixed train-validation-test splits. The obtained results confirm the viability of the proposed datasets, achieving the best overall result of 81.74\% F1-score for bimodal classification.