San-BERT: Extractive Summarization for Sanskrit Documents using BERT and it's variants

TL;DR

This paper develops Sanskrit language models based on BERT and its variants, and uses them with clustering techniques to generate extractive summaries for Sanskrit documents, also releasing a new Sanskrit corpus.

Contribution

Introduces Sanskrit-specific BERT models and a new Devanagari Sanskrit corpus, applying feature extraction, dimensionality reduction, and clustering for extractive summarization.

Findings

Effective Sanskrit BERT models developed

Successful extractive summarization demonstrated

Public Sanskrit corpus released

Abstract

In this work, we develop language models for the Sanskrit language, namely Bidirectional Encoder Representations from Transformers (BERT) and its variants: A Lite BERT (ALBERT), and Robustly Optimized BERT (RoBERTa) using Devanagari Sanskrit text corpus. Then we extracted the features for the given text from these models. We applied the dimensional reduction and clustering techniques on the features to generate an extractive summary for a given Sanskrit document. Along with the extractive text summarization techniques, we have also created and released a Sanskrit Devanagari text corpus publicly.