Lyric document embeddings for music tagging

TL;DR

This paper investigates various methods for creating fixed-dimensional lyric embeddings for music tagging, finding that simple averaging of pretrained embeddings often outperforms complex neural models across multiple tasks.

Contribution

It provides an extensive empirical comparison of token-level and document-level lyric embedding methods on a large-scale dataset for music tagging.

Findings

Averaging pretrained embeddings outperforms complex neural architectures in many tasks.

Simple methods are competitive with or better than advanced models.

The study covers diverse tagging tasks like genre, explicit content, and era detection.

Abstract

We present an empirical study on embedding the lyrics of a song into a fixed-dimensional feature for the purpose of music tagging. Five methods of computing token-level and four methods of computing document-level representations are trained on an industrial-scale dataset of tens of millions of songs. We compare simple averaging of pretrained embeddings to modern recurrent and attention-based neural architectures. Evaluating on a wide range of tagging tasks such as genre classification, explicit content identification and era detection, we find that averaging word embeddings outperform more complex architectures in many downstream metrics.