An Improved Historical Embedding without Alignment

TL;DR

This paper introduces a scalable, alignment-free method for encoding historical word meanings into a single dense vector space, improving the detection of semantic change over time.

Contribution

The paper presents a novel approach that eliminates the need for costly embedding alignment across time periods, enhancing efficiency and accuracy in modeling semantic evolution.

Findings

Outperformed three popular methods in identifying semantic change

Effective visualization of semantic evolution of words

Scalable and computationally efficient approach

Abstract

Many words have evolved in meaning as a result of cultural and social change. Understanding such changes is crucial for modelling language and cultural evolution. Low-dimensional embedding methods have shown promise in detecting words' meaning change by encoding them into dense vectors. However, when exploring semantic change of words over time, these methods require the alignment of word embeddings across different time periods. This process is computationally expensive, prohibitively time consuming and suffering from contextual variability. In this paper, we propose a new and scalable method for encoding words from different time periods into one dense vector space. This can greatly improve performance when it comes to identifying words that have changed in meaning over time. We evaluated our method on dataset from Google Books N-gram. Our method outperformed three other popular methods in terms of the number of words correctly identified to have changed in meaning. Additionally, we provide an intuitive visualization of the semantic evolution of some words extracted by our method