Chandomitra: Towards Generating Structured Sanskrit Poetry from Natural Language Inputs

TL;DR

This paper introduces Chandomitra, a dataset and methodology for generating structured Sanskrit poetry from English inputs, leveraging large language models with specialized techniques to balance syntactic accuracy and poetic quality.

Contribution

It presents a new dataset and benchmarks for Sanskrit poetry generation, along with techniques like constrained decoding and instruction fine-tuning to improve output quality.

Findings

Constrained decoding achieves 99.86% syntactic accuracy.

Instruction fine-tuning enhances semantic coherence and poetic qualities.

Models outperform GPT-4o in specific metrics.

Abstract

Text Generation has achieved remarkable performance using large language models. It has also been recently well-studied that these large language models are capable of creative generation tasks but prominently for high-resource languages. This prompts a fundamental question: Is there a way to utilize these (large) language models for structured poetry generation in a low-resource language, such as Sanskrit? We present Chandomitra, an English input to structured Sanskrit Poetry translation dataset, specifically adhering to the Anushtubh meter. We benchmark various open and closed models, and scrutinize specialized techniques such as constrained decoding and instruction fine-tuning, for the proposed task. Our constrained decoding methodology achieves 99.86% syntactic accuracy in generating metrically valid Sanskrit poetry, outperforming GPT-4o (1-shot: 31.24%). Our best-performing instruction-tuned model, on the other hand, performs better in semantic coherence with the English input, at the expense of slightly lower syntactic accuracy. Human evaluation further reveals that instruction fine-tuned model is better able to capture the poetic aspects. Data and Code are available.