LyricSIM: A novel Dataset and Benchmark for Similarity Detection in Spanish Song LyricS

TL;DR

This paper introduces LyricSIM, a new high-quality dataset and benchmark for measuring semantic similarity in Spanish song lyrics, facilitating future research and applications in this domain.

Contribution

The paper provides a novel dataset of 676 annotated Spanish song lyric pairs and establishes baseline performance for similarity detection models.

Findings

High inter-annotator agreement achieved

Baseline models show promising results on the dataset

Dataset enables future research in Spanish lyric similarity

Abstract

In this paper, we present a new dataset and benchmark tailored to the task of semantic similarity in song lyrics. Our dataset, originally consisting of 2775 pairs of Spanish songs, was annotated in a collective annotation experiment by 63 native annotators. After collecting and refining the data to ensure a high degree of consensus and data integrity, we obtained 676 high-quality annotated pairs that were used to evaluate the performance of various state-of-the-art monolingual and multilingual language models. Consequently, we established baseline results that we hope will be useful to the community in all future academic and industrial applications conducted in this context.