Tracking Semantic Change in Slovene: A Novel Dataset and Optimal Transport-Based Distance

TL;DR

This paper introduces a new Slovene dataset for semantic change detection, analyzes existing metrics, and proposes an optimal transport-based method that improves the robustness and accuracy of detecting semantic shifts.

Contribution

It provides the first Slovene semantic change dataset and develops a novel optimal transport-based metric for more effective semantic change detection.

Findings

Optimal transport metric outperforms existing measures

New dataset enables evaluation of semantic change methods

Proposed approach achieves comparable or better results

Abstract

In this paper, we focus on the detection of semantic changes in Slovene, a less resourced Slavic language with two million speakers. Detecting and tracking semantic changes provides insight into the evolution of language caused by changes in society and culture. We present the first Slovene dataset for evaluating semantic change detection systems, which contains aggregated semantic change scores for 104 target words obtained from more than 3,000 manually annotated sentence pairs. We analyze an important class of measures of semantic change metrics based on the Average pairwise distance and identify several limitations. To address these limitations, we propose a novel metric based on regularized optimal transport, which offers a more robust framework for quantifying semantic change. We provide a comprehensive evaluation of various existing semantic change detection methods and associated semantic change measures on our dataset. Through empirical testing, we demonstrate that our proposed approach, leveraging regularized optimal transport, achieves either matching or improved performance compared to baseline approaches.