TransWiC at SemEval-2021 Task 2: Transformer-based Multilingual and Cross-lingual Word-in-Context Disambiguation

TL;DR

This paper presents a transformer-based approach for multilingual and cross-lingual word-in-context disambiguation that avoids language-specific resources, achieving high accuracy across various language pairs.

Contribution

It introduces a language-resource-independent method using pretrained transformers for word sense disambiguation in multiple languages.

Findings

Achieved 0.90 accuracy in English-English subtask

Comparable results to state-of-the-art with no language-specific resources

Effective in both monolingual and cross-lingual settings

Abstract

Identifying whether a word carries the same meaning or different meaning in two contexts is an important research area in natural language processing which plays a significant role in many applications such as question answering, document summarisation, information retrieval and information extraction. Most of the previous work in this area rely on language-specific resources making it difficult to generalise across languages. Considering this limitation, our approach to SemEval-2021 Task 2 is based only on pretrained transformer models and does not use any language-specific processing and resources. Despite that, our best model achieves 0.90 accuracy for English-English subtask which is very compatible compared to the best result of the subtask; 0.93 accuracy. Our approach also achieves satisfactory results in other monolingual and cross-lingual language pairs as well.