Combating the Curse of Multilinguality in Cross-Lingual WSD by Aligning Sparse Contextualized Word Representations

TL;DR

This paper proposes a method to improve cross-lingual zero-shot word sense disambiguation by aligning sparse contextualized representations from monolingual models, achieving significant accuracy gains across diverse languages.

Contribution

It introduces a novel approach combining large monolingual models with sparse representations and a contextualized mapping, enhancing cross-lingual WSD performance.

Findings

6.5-point increase in F-score over baseline

Effective across 17 diverse languages

Open-source code available for replication

Abstract

In this paper, we advocate for using large pre-trained monolingual language models in cross lingual zero-shot word sense disambiguation (WSD) coupled with a contextualized mapping mechanism. We also report rigorous experiments that illustrate the effectiveness of employing sparse contextualized word representations obtained via a dictionary learning procedure. Our experimental results demonstrate that the above modifications yield a significant improvement of nearly 6.5 points of increase in the average F-score (from 62.0 to 68.5) over a collection of 17 typologically diverse set of target languages. We release our source code for replicating our experiments at https://github.com/begab/sparsity_makes_sense.