SenWiCh: Sense-Annotation of Low-Resource Languages for WiC using Hybrid Methods

TL;DR

This paper introduces new sense-annotated datasets for ten low-resource languages to improve cross-lingual transfer and polysemy disambiguation, utilizing a semi-automatic annotation method and WiC-based evaluation.

Contribution

It presents the first sense-annotated datasets for low-resource languages and a semi-automatic annotation approach to facilitate dataset creation.

Findings

Datasets enable better cross-lingual transfer evaluation.

Semi-automatic annotation improves efficiency and quality.

Results show the importance of targeted datasets for polysemy disambiguation.

Abstract

This paper addresses the critical need for high-quality evaluation datasets in low-resource languages to advance cross-lingual transfer. While cross-lingual transfer offers a key strategy for leveraging multilingual pretraining to expand language technologies to understudied and typologically diverse languages, its effectiveness is dependent on quality and suitable benchmarks. We release new sense-annotated datasets of sentences containing polysemous words, spanning ten low-resource languages across diverse language families and scripts. To facilitate dataset creation, the paper presents a demonstrably beneficial semi-automatic annotation method. The utility of the datasets is demonstrated through Word-in-Context (WiC) formatted experiments that evaluate transfer on these low-resource languages. Results highlight the importance of targeted dataset creation and evaluation for effective polysemy disambiguation in low-resource settings and transfer studies. The released datasets and code aim to support further research into fair, robust, and truly multilingual NLP.