Leveraging Contextual Embeddings for Detecting Diachronic Semantic Shift

TL;DR

This paper introduces a method using contextual embeddings from BERT to detect diachronic semantic shifts across time, domains, and languages, showing comparable or promising results without extensive domain adaptation.

Contribution

It presents a novel approach leveraging BERT-based contextual embeddings for diachronic semantic shift detection, applicable across domains and languages, with efficient performance.

Findings

Performance comparable to state-of-the-art without domain adaptation

Effective in detecting short-term yearly semantic shifts

Shows promising results in multilingual semantic shift detection

Abstract

We propose a new method that leverages contextual embeddings for the task of diachronic semantic shift detection by generating time specific word representations from BERT embeddings. The results of our experiments in the domain specific LiverpoolFC corpus suggest that the proposed method has performance comparable to the current state-of-the-art without requiring any time consuming domain adaptation on large corpora. The results on the newly created Brexit news corpus suggest that the method can be successfully used for the detection of a short-term yearly semantic shift. And lastly, the model also shows promising results in a multilingual settings, where the task was to detect differences and similarities between diachronic semantic shifts in different languages.