Unsupervised detection of diachronic word sense evolution

TL;DR

This paper introduces a fast, linear method for tracking how word meanings change over time using time series of word embeddings, enabling real-time diachronic semantic analysis of social media data.

Contribution

It presents a novel linear approach to create congruent temporal word embeddings, allowing real-time detection of semantic shifts and biases across different time periods.

Findings

Effective tracking of semantic drift in words like 'amazon' and 'apple' over time.

Ability to analyze gender bias and concept shifts in real-time social media streams.

Method is scalable and suitable for distributed processing.

Abstract

Most words have several senses and connotations which evolve in time due to semantic shift, so that closely related words may gain different or even opposite meanings over the years. This evolution is very relevant to the study of language and of cultural changes, but the tools currently available for diachronic semantic analysis have significant, inherent limitations and are not suitable for real-time analysis. In this article, we demonstrate how the linearity of random vectors techniques enables building time series of congruent word embeddings (or semantic spaces) which can then be compared and combined linearly without loss of precision over any time period to detect diachronic semantic shifts. We show how this approach yields time trajectories of polysemous words such as amazon or apple, enables following semantic drifts and gender bias across time, reveals the shifting instantiations of stable concepts such as hurricane or president. This very fast, linear approach can easily be distributed over many processors to follow in real time streams of social media such as Twitter or Facebook; the resulting, time-dependent semantic spaces can then be combined at will by simple additions or subtractions.