Automated Evaluation of Meter and Rhyme in Russian Generative and Human-Authored Poetry

TL;DR

This paper introduces tools and datasets for evaluating Russian poetry's metrical and rhyming qualities, aiding the development of generative AI in poetry creation.

Contribution

It presents the Russian Poetry Scansion Tool for stress and rhyme analysis and the RIFMA dataset for stress annotation in poetic fragments.

Findings

The tools effectively identify stress and rhyme patterns in Russian poetry.

The RIFMA dataset enables evaluation of language models' stress placement accuracy.

Resources support advancements in generative poetry AI systems.

Abstract

Generative poetry systems require effective tools for data engineering and automatic evaluation, particularly to assess how well a poem adheres to versification rules, such as the correct alternation of stressed and unstressed syllables and the presence of rhymes. In this work, we introduce the Russian Poetry Scansion Tool library designed for stress mark placement in Russian-language syllabo-tonic poetry, rhyme detection, and identification of defects of poeticness. Additionally, we release RIFMA -- a dataset of poem fragments spanning various genres and forms, annotated with stress marks. This dataset can be used to evaluate the capability of modern large language models to accurately place stress marks in poetic texts. The published resources provide valuable tools for researchers and practitioners in the field of creative generative AI, facilitating advancements in the development and evaluation of generative poetry systems.