Linguistically-Informed Neural Architectures for Lexical, Syntactic and Semantic Tasks in Sanskrit

TL;DR

This paper develops linguistically-informed neural models for Sanskrit NLP tasks, addressing challenges like morphology and low-resource constraints, and introduces a toolkit to enhance accessibility of Sanskrit manuscripts.

Contribution

It proposes novel neural architectures tailored for Sanskrit, demonstrating state-of-the-art results and providing a web-based toolkit for real-time linguistic analysis.

Findings

Achieved state-of-the-art performance in Sanskrit NLP tasks

Developed a web-based toolkit for real-time Sanskrit analysis

Enhanced accessibility of Sanskrit manuscripts through NLP technologies

Abstract

The primary focus of this thesis is to make Sanskrit manuscripts more accessible to the end-users through natural language technologies. The morphological richness, compounding, free word orderliness, and low-resource nature of Sanskrit pose significant challenges for developing deep learning solutions. We identify four fundamental tasks, which are crucial for developing a robust NLP technology for Sanskrit: word segmentation, dependency parsing, compound type identification, and poetry analysis. The first task, Sanskrit Word Segmentation (SWS), is a fundamental text processing task for any other downstream applications. However, it is challenging due to the sandhi phenomenon that modifies characters at word boundaries. Similarly, the existing dependency parsing approaches struggle with morphologically rich and low-resource languages like Sanskrit. Compound type identification is also challenging for Sanskrit due to the context-sensitive semantic relation between components. All these challenges result in sub-optimal performance in NLP applications like question answering and machine translation. Finally, Sanskrit poetry has not been extensively studied in computational linguistics. While addressing these challenges, this thesis makes various contributions: (1) The thesis proposes linguistically-informed neural architectures for these tasks. (2) We showcase the interpretability and multilingual extension of the proposed systems. (3) Our proposed systems report state-of-the-art performance. (4) Finally, we present a neural toolkit named SanskritShala, a web-based application that provides real-time analysis of input for various NLP tasks. Overall, this thesis contributes to making Sanskrit manuscripts more accessible by developing robust NLP technology and releasing various resources, datasets, and web-based toolkit.