The challenges of temporal alignment on Twitter during crises

TL;DR

This paper investigates the rapid language changes in social media during crises and proposes domain adaptation techniques to improve NLP model performance amidst temporal degradation.

Contribution

It introduces the first analysis of adversarial-driven language change during crises and develops methods for temporal adaptation in short-term crisis data.

Findings

Temporal degradation significantly impacts NLP performance during crises.

Domain adaptation methods can mitigate performance loss in short-term crisis scenarios.

Current methods face limitations when unlabeled data is scarce.

Abstract

Language use changes over time, and this impacts the effectiveness of NLP systems. This phenomenon is even more prevalent in social media data during crisis events where meaning and frequency of word usage may change over the course of days. Contextual language models fail to adapt temporally, emphasizing the need for temporal adaptation in models which need to be deployed over an extended period of time. While existing approaches consider data spanning large periods of time (from years to decades), shorter time spans are critical for crisis data. We quantify temporal degradation for this scenario and propose methods to cope with performance loss by leveraging techniques from domain adaptation. To the best of our knowledge, this is the first effort to explore effects of rapid language change driven by adversarial adaptations, particularly during natural and human-induced disasters. Through extensive experimentation on diverse crisis datasets, we analyze under what conditions our approaches outperform strong baselines while highlighting the current limitations of temporal adaptation methods in scenarios where access to unlabeled data is scarce.