SinFoS: A Parallel Dataset for Translating Sinhala Figures of Speech

TL;DR

This paper introduces SinFoS, a new dataset of Sinhala figures of speech with annotations, to improve translation and understanding of culturally rooted expressions in low-resource NLP contexts.

Contribution

The paper presents a novel dataset of 2,344 Sinhala FoS with annotations, along with classification tools and evaluation of LLMs, addressing low-resource and cultural challenges in machine translation.

Findings

Binary classifier achieved 92% accuracy in distinguishing FoS types.

Current LLMs struggle with accurately translating idiomatic expressions.

The dataset serves as a benchmark for low-resource, culturally-aware NLP research.

Abstract

Figures of Speech (FoS) consist of multi-word phrases that are deeply intertwined with culture. While Neural Machine Translation (NMT) performs relatively well with the figurative expressions of high-resource languages, it often faces challenges when dealing with low-resource languages like Sinhala due to limited available data. To address this limitation, we introduce a corpus of 2,344 Sinhala figures of speech with cultural and cross-lingual annotations. We examine this dataset to classify the cultural origins of the figures of speech and to identify their cross-lingual equivalents. Additionally, we have developed a binary classifier to differentiate between two types of FOS in the dataset, achieving an accuracy rate of approximately 92%. We also evaluate the performance of existing LLMs on this dataset. Our findings reveal significant shortcomings in the current capabilities of LLMs, as these models often struggle to accurately convey idiomatic meanings. By making this dataset publicly available, we offer a crucial benchmark for future research in low-resource NLP and culturally aware machine translation.