Utilizing Wordnets for Cognate Detection among Indian Languages

TL;DR

This paper presents a deep learning approach to detect cognate pairs among ten Indian languages using resources like IndoWordnet and parallel corpora, improving detection accuracy and providing valuable datasets for NLP applications.

Contribution

It introduces a neural network-based method leveraging Wordnets and parallel corpora for cognate detection among Indian languages, with improved performance and new datasets.

Findings

Up to 26% performance improvement in cognate detection.

Effective use of IndoWordnet and parallel corpora resources.

Release of datasets of detected cognates for Indian languages.

Abstract

Automatic Cognate Detection (ACD) is a challenging task which has been utilized to help NLP applications like Machine Translation, Information Retrieval and Computational Phylogenetics. Unidentified cognate pairs can pose a challenge to these applications and result in a degradation of performance. In this paper, we detect cognate word pairs among ten Indian languages with Hindi and use deep learning methodologies to predict whether a word pair is cognate or not. We identify IndoWordnet as a potential resource to detect cognate word pairs based on orthographic similarity-based methods and train neural network models using the data obtained from it. We identify parallel corpora as another potential resource and perform the same experiments for them. We also validate the contribution of Wordnets through further experimentation and report improved performance of up to 26%. We discuss the nuances of cognate detection among closely related Indian languages and release the lists of detected cognates as a dataset. We also observe the behaviour of, to an extent, unrelated Indian language pairs and release the lists of detected cognates among them as well.