TRBLLmaker -- Transformer Reads Between Lyrics Lines maker

TL;DR

This paper introduces a generative Transformer-based model to produce implicit song meanings from lyrics, comparing GPT-2 and T5 architectures, and evaluating various prompts and decoding methods.

Contribution

It presents a novel approach to generate song interpretations using Transformer models, incorporating prompt variations and decoding strategies, with evaluation on a new dataset.

Findings

GPT-2 outperforms T5 in ROUGE scores

Prompt design significantly affects output quality

Decoding method impacts the relevance of generated meanings

Abstract

Even for us, it can be challenging to comprehend the meaning of songs. As part of this project, we explore the process of generating the meaning of songs. Despite the widespread use of text-to-text models, few attempts have been made to achieve a similar objective. Songs are primarily studied in the context of sentiment analysis. This involves identifying opinions and emotions in texts, evaluating them as positive or negative, and utilizing these evaluations to make music recommendations. In this paper, we present a generative model that offers implicit meanings for several lines of a song. Our model uses a decoder Transformer architecture GPT-2, where the input is the lyrics of a song. Furthermore, we compared the performance of this architecture with that of the encoder-decoder Transformer architecture of the T5 model. We also examined the effect of different prompt types with the option of appending additional information, such as the name of the artist and the title of the song. Moreover, we tested different decoding methods with different training parameters and evaluated our results using ROUGE. In order to build our dataset, we utilized the 'Genious' API, which allowed us to acquire the lyrics of songs and their explanations, as well as their rich metadata.