A Data Bootstrapping Recipe for Low Resource Multilingual Relation Classification

TL;DR

This paper introduces IndoRE, a new multilingual relation classification dataset for Indian languages and English, and investigates transfer learning and data augmentation techniques to improve low-resource multilingual relation classification.

Contribution

It provides a large, annotated dataset for Indian languages and analyzes transfer learning strategies and data quality tradeoffs for relation classification.

Findings

mBERT-based system achieves competitive results

Translation and alignment improve low-resource performance

Silver data can be a cost-effective alternative to gold annotations

Abstract

Relation classification (sometimes called 'extraction') requires trustworthy datasets for fine-tuning large language models, as well as for evaluation. Data collection is challenging for Indian languages, because they are syntactically and morphologically diverse, as well as different from resource-rich languages like English. Despite recent interest in deep generative models for Indian languages, relation classification is still not well served by public data sets. In response, we present IndoRE, a dataset with 21K entity and relation tagged gold sentences in three Indian languages, plus English. We start with a multilingual BERT (mBERT) based system that captures entity span positions and type information and provides competitive monolingual relation classification. Using this system, we explore and compare transfer mechanisms between languages. In particular, we study the accuracy efficiency tradeoff between expensive gold instances vs. translated and aligned 'silver' instances. We release the dataset for future research.