Aligning Language Models for Lyric-to-Melody Generation with Rule-Based Musical Constraints

TL;DR

This paper introduces a novel alignment framework for lyric-to-melody generation that reduces rule violations and improves musicality by automatically generating preference data from rule-based constraints and applying sequential preference optimization.

Contribution

The work presents a new alignment method that enhances LLM-based melody generation without human annotation, combining rule-based constraints with preference optimization techniques.

Findings

Significantly reduces rule violations in generated melodies.

Outperforms baselines in objective and subjective evaluations.

Improves musical coherence and naturalness of generated melodies.

Abstract

Large Language Models (LLMs) show promise in lyric-to-melody generation, but models trained with Supervised Fine-Tuning (SFT) often produce musically implausible melodies with issues like poor rhythm and unsuitable vocal ranges, a phenomenon we term "constraint violation". To address this, we propose a novel alignment framework that instills musical knowledge without human annotation. We define rule-based musical constraints to automatically generate a preference dataset from an SFT model's outputs. The model is then aligned through a sequential process, first using Direct Preference Optimization (DPO) on paired preference data, followed by Kahneman-Tversky Optimization (KTO) on unpaired negative samples. Experimental results demonstrate that our aligned model substantially reduces rule violations and outperforms strong baselines in both objective and subjective evaluations, generating melodies with substantially improved musicality and coherence. An interactive demo with audio comparisons is available at https://arain233.github.io/AligningMelody-demo.