Encoder-Decoder Framework for Interactive Free Verses with Generation with Controllable High-Quality Rhyming

TL;DR

This paper introduces a novel fine-tuning method for lyric generation that improves rhyming accuracy and readability by prepending rhyming words, leveraging pretrained language models and supporting multilingual applications.

Contribution

The paper presents a new fine-tuning approach that enables better rhyming control in lyric generation while maintaining compatibility with pretrained language models.

Findings

The proposed method outperforms state-of-the-art strategies in rhyming quality.

It produces more readable and coherent lyrics with improved rhyming.

The approach is effective across multiple languages, demonstrating multilingual viability.

Abstract

Composing poetry or lyrics involves several creative factors, but a challenging aspect of generation is the adherence to a more or less strict metric and rhyming pattern. To address this challenge specifically, previous work on the task has mainly focused on reverse language modeling, which brings the critical selection of each rhyming word to the forefront of each verse. On the other hand, reversing the word order requires that models be trained from scratch with this task-specific goal and cannot take advantage of transfer learning from a Pretrained Language Model (PLM). We propose a novel fine-tuning approach that prepends the rhyming word at the start of each lyric, which allows the critical rhyming decision to be made before the model commits to the content of the lyric (as during reverse language modeling), but maintains compatibility with the word order of regular PLMs as the lyric itself is still generated in left-to-right order. We conducted extensive experiments to compare this fine-tuning against the current state-of-the-art strategies for rhyming, finding that our approach generates more readable text and better rhyming capabilities. Furthermore, we furnish a high-quality dataset in English and 12 other languages, analyse the approach's feasibility in a multilingual context, provide extensive experimental results shedding light on good and bad practices for lyrics generation, and propose metrics to compare methods in the future.