On the Linearity of Semantic Change: Investigating Meaning Variation via Dynamic Graph Models

TL;DR

This paper introduces two graph-based models to analyze semantic change over time, revealing that meaning variation follows linear patterns in embedding space and self-similarity decay across three languages.

Contribution

It proposes novel linear models for semantic change and demonstrates their validity across multiple languages and datasets.

Findings

Semantic change is linear in embedding space.

Self-similarity of words decays linearly over time.

Models apply successfully to multilingual corpora.

Abstract

We consider two graph models of semantic change. The first is a time-series model that relates embedding vectors from one time period to embedding vectors of previous time periods. In the second, we construct one graph for each word: nodes in this graph correspond to time points and edge weights to the similarity of the word's meaning across two time points. We apply our two models to corpora across three different languages. We find that semantic change is linear in two senses. Firstly, today's embedding vectors (= meaning) of words can be derived as linear combinations of embedding vectors of their neighbors in previous time periods. Secondly, self-similarity of words decays linearly in time. We consider both findings as new laws/hypotheses of semantic change.