REFFLY: Melody-Constrained Lyrics Editing Model
Songyan Zhao, Bingxuan Li, Yufei Tian, Nanyun Peng

TL;DR
REFFLY is a novel lyric revision framework that transforms plain text into melody-aligned lyrics, enabling flexible applications like translation and style transfer, and outperforms existing models in musicality and quality.
Contribution
This paper introduces REFFLY, the first revision framework for melody-aligned lyrics, trained on a curated dataset and enhanced with training-free heuristics for semantic and musical consistency.
Findings
REFFLY outperforms Lyra and GPT-4 by 25% in musicality.
Effective in tasks like lyrics generation and song translation.
Demonstrates improved semantic preservation and musical alignment.
Abstract
Automatic melody-to-lyric (M2L) generation aims to create lyrics that align with a given melody. While most previous approaches generate lyrics from scratch, revision, editing plain text draft to fit it into the melody, offers a much more flexible and practical alternative. This enables broad applications, such as generating lyrics from flexible inputs (keywords, themes, or full text that needs refining to be singable), song translation (preserving meaning across languages while keeping the melody intact), or style transfer (adapting lyrics to different genres). This paper introduces REFFLY (REvision Framework For LYrics), the first revision framework for editing and generating melody-aligned lyrics. We train the lyric revision module using our curated synthesized melody-aligned lyrics dataset, enabling it to transform plain text into lyrics that align with a given melody. To further…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
Taxonomy
TopicsMusic and Audio Processing · Music Technology and Sound Studies
MethodsAttention Is All You Need · Byte Pair Encoding · Absolute Position Encodings · Softmax · Label Smoothing · Dropout · Layer Normalization · Position-Wise Feed-Forward Layer · Linear Layer · Adam
