Melody-Lyrics Matching with Contrastive Alignment Loss

TL;DR

This paper introduces a new melody-lyrics matching task using a contrastive alignment loss, enabling retrieval of lyrics for melodies without needing alignment annotations, leveraging self-supervised learning and a novel syllable-level lyric representation.

Contribution

It proposes a self-supervised contrastive learning framework for melody-lyrics matching that does not require alignment annotations and introduces a new syllable-level lyric representation.

Findings

The method effectively matches melodies with coherent lyrics.

It leverages existing paired melody-lyrics data without annotations.

Empirical results demonstrate the approach's potential.

Abstract

The connection between music and lyrics is far beyond semantic bonds. Conceptual pairs in the two modalities such as rhythm and rhyme, note duration and syllabic stress, and structure correspondence, raise a compelling yet seldom-explored direction in the field of music information retrieval. In this paper, we present melody-lyrics matching (MLM), a new task which retrieves potential lyrics for a given symbolic melody from text sources. Rather than generating lyrics from scratch, MLM essentially exploits the relationships between melody and lyrics. We propose a self-supervised representation learning framework with contrastive alignment loss for melody and lyrics. This has the potential to leverage the abundance of existing songs with paired melody and lyrics. No alignment annotations are required. Additionally, we introduce sylphone, a novel representation for lyrics at syllable-level activated by phoneme identity and vowel stress. We demonstrate that our method can match melody with coherent and singable lyrics with empirical results and intuitive examples. We open source code and provide matching examples on the companion webpage: https://github.com/changhongw/mlm.