Opinions are Made to be Changed: Temporally Adaptive Stance Classification

TL;DR

This paper addresses the challenge of declining stance classification performance over time due to evolving language in social media, proposing a temporal adaptation method for word embeddings to maintain classifier accuracy.

Contribution

It introduces large-scale longitudinal stance datasets and a novel incremental temporal alignment approach to adapt embeddings over time, improving classifier robustness.

Findings

Embedding adaptation reduces performance decay over time.

Incremental Temporal Alignment outperforms other adaptation methods.

Temporal adaptation enables use of unlabelled data for current language trends.

Abstract

Given the rapidly evolving nature of social media and people's views, word usage changes over time. Consequently, the performance of a classifier trained on old textual data can drop dramatically when tested on newer data. While research in stance classification has advanced in recent years, no effort has been invested in making these classifiers have persistent performance over time. To study this phenomenon we introduce two novel large-scale, longitudinal stance datasets. We then evaluate the performance persistence of stance classifiers over time and demonstrate how it decays as the temporal gap between training and testing data increases. We propose a novel approach to mitigate this performance drop, which is based on temporal adaptation of the word embeddings used for training the stance classifier. This enables us to make use of readily available unlabelled data from the current time period instead of expensive annotation efforts. We propose and compare several approaches to embedding adaptation and find that the Incremental Temporal Alignment (ITA) model leads to the best results in reducing performance drop over time.