Mono vs Multilingual BERT: A Case Study in Hindi and Marathi Named Entity Recognition

TL;DR

This study compares monolingual and multilingual transformer models for NER in Hindi and Marathi, revealing that monolingual MahaRoBERTa excels in Marathi while multilingual XLM-RoBERTa is best for Hindi, with insights from cross-language evaluation.

Contribution

It provides an exhaustive benchmark of transformer models for Hindi and Marathi NER, highlighting the performance differences between monolingual and multilingual models.

Findings

MahaRoBERTa outperforms others in Marathi NER.

XLM-RoBERTa performs best for Hindi NER.

Cross-language evaluation shows mixed results.

Abstract

Named entity recognition (NER) is the process of recognising and classifying important information (entities) in text. Proper nouns, such as a person's name, an organization's name, or a location's name, are examples of entities. The NER is one of the important modules in applications like human resources, customer support, search engines, content classification, and academia. In this work, we consider NER for low-resource Indian languages like Hindi and Marathi. The transformer-based models have been widely used for NER tasks. We consider different variations of BERT like base-BERT, RoBERTa, and AlBERT and benchmark them on publicly available Hindi and Marathi NER datasets. We provide an exhaustive comparison of different monolingual and multilingual transformer-based models and establish simple baselines currently missing in the literature. We show that the monolingual MahaRoBERTa model performs the best for Marathi NER whereas the multilingual XLM-RoBERTa performs the best for Hindi NER. We also perform cross-language evaluation and present mixed observations.