Harnessing Cross-lingual Features to Improve Cognate Detection for Low-resource Languages

TL;DR

This paper presents a novel approach using cross-lingual embeddings and knowledge graph context to improve cognate detection among Indian languages, significantly enhancing downstream machine translation performance.

Contribution

It introduces a new method combining cross-lingual embeddings with knowledge graph context for better cognate detection in low-resource languages.

Findings

Up to 18% improvement in cognate detection F-score.

Cognate detection enhances NMT BLEU scores by up to 2.76.

Created new datasets for Indian languages.

Abstract

Cognates are variants of the same lexical form across different languages; for example 'fonema' in Spanish and 'phoneme' in English are cognates, both of which mean 'a unit of sound'. The task of automatic detection of cognates among any two languages can help downstream NLP tasks such as Cross-lingual Information Retrieval, Computational Phylogenetics, and Machine Translation. In this paper, we demonstrate the use of cross-lingual word embeddings for detecting cognates among fourteen Indian Languages. Our approach introduces the use of context from a knowledge graph to generate improved feature representations for cognate detection. We, then, evaluate the impact of our cognate detection mechanism on neural machine translation (NMT), as a downstream task. We evaluate our methods to detect cognates on a challenging dataset of twelve Indian languages, namely, Sanskrit, Hindi, Assamese, Oriya, Kannada, Gujarati, Tamil, Telugu, Punjabi, Bengali, Marathi, and Malayalam. Additionally, we create evaluation datasets for two more Indian languages, Konkani and Nepali. We observe an improvement of up to 18% points, in terms of F-score, for cognate detection. Furthermore, we observe that cognates extracted using our method help improve NMT quality by up to 2.76 BLEU. We also release our code, newly constructed datasets and cross-lingual models publicly.