XL-DURel: Finetuning Sentence Transformers for Ordinal Word-in-Context Classification

TL;DR

This paper introduces XL-DURel, a multilingual Sentence Transformer fine-tuned for ordinal Word-in-Context classification, demonstrating improved performance through novel loss functions and a unified approach to WiC tasks.

Contribution

The paper presents a new multilingual Sentence Transformer model optimized for ordinal classification, with a ranking-based loss function, unifying WiC task formulations.

Findings

Outperforms previous models on ordinal and binary WiC data.

Ranking objective based on angular distance improves classification.

Unified treatment of binary and ordinal WiC tasks enhances performance.

Abstract

We propose XL-DURel, a finetuned, multilingual Sentence Transformer model optimized for ordinal Word-in-Context classification. We test several loss functions for regression and ranking tasks managing to outperform previous models on ordinal and binary data with a ranking objective based on angular distance in complex space. We further show that binary WiC can be treated as a special case of ordinal WiC and that optimizing models for the general ordinal task improves performance on the more specific binary task. This paves the way for a unified treatment of WiC modeling across different task formulations.