RuSemShift: a dataset of historical lexical semantic change in Russian

TL;DR

RuSemShift is a large, manually annotated dataset for studying lexical semantic change in Russian across two historical periods, enabling evaluation of computational models in this domain.

Contribution

This paper introduces RuSemShift, the first large-scale, annotated dataset for Russian semantic change, and evaluates baseline distributional models on it.

Findings

Distributional models show promising results on RuSemShift

Crowd-sourced annotations are consistent and reliable

The dataset covers two significant historical periods in Russian

Abstract

We present RuSemShift, a large-scale manually annotated test set for the task of semantic change modeling in Russian for two long-term time period pairs: from the pre-Soviet through the Soviet times and from the Soviet through the post-Soviet times. Target words were annotated by multiple crowd-source workers. The annotation process was organized following the DURel framework and was based on sentence contexts extracted from the Russian National Corpus. Additionally, we report the performance of several distributional approaches on RuSemShift, achieving promising results, which at the same time leave room for other researchers to improve.