Temporal Word Meaning Disambiguation using TimeLMs

TL;DR

This paper investigates the use of time-aware language models to improve word sense disambiguation by capturing semantic changes over time, addressing limitations of static embeddings.

Contribution

It introduces methods leveraging TimeLMs for temporal word sense disambiguation and provides ablation studies demonstrating their effectiveness.

Findings

TimeLMs improve disambiguation accuracy

Ablation studies identify key factors for success

Future directions for temporal semantics are discussed

Abstract

Meaning of words constantly changes given the events in modern civilization. Large Language Models use word embeddings, which are often static and thus cannot cope with this semantic change. Thus,it is important to resolve ambiguity in word meanings. This paper is an effort in this direction, where we explore methods for word sense disambiguation for the EvoNLP shared task. We conduct rigorous ablations for two solutions to this problem. We see that an approach using time-aware language models helps this task. Furthermore, we explore possible future directions to this problem.