COVID-19 South African Vaccine Hesitancy Models Show Boost in Performance Upon Fine-Tuning on M-pox Tweets

TL;DR

This study demonstrates that fine-tuning COVID-19 sentiment analysis models on M-pox tweets significantly improves their performance, achieving near 70% F1-score and outperforming existing models.

Contribution

The paper shows that adapting COVID-19 models to M-pox data enhances classification accuracy and provides insights for developing more sophisticated models using topic analysis.

Findings

Fine-tuning increased F1-score by over 8%.

Fine-tuned models outperform state-of-the-art classifiers.

LDA topic modeling reveals insights into misclassified tweets.

Abstract

Very large numbers of M-pox cases have, since the start of May 2022, been reported in non-endemic countries leading many to fear that the M-pox Outbreak would rapidly transition into another pandemic, while the COVID-19 pandemic ravages on. Given the similarities of M-pox with COVID-19, we chose to test the performance of COVID-19 models trained on South African twitter data on a hand-labelled M-pox dataset before and after fine-tuning. More than 20k M-pox-related tweets from South Africa were hand-labelled as being either positive, negative or neutral. After fine-tuning these COVID-19 models on the M-pox dataset, the F1-scores increased by more than 8% falling just short of 70%, but still outperforming state-of-the-art models and well-known classification algorithms. An LDA-based topic modelling procedure was used to compare the miss-classified M-pox tweets of the original COVID-19 RoBERTa model with its fine-tuned version, and from this analysis, we were able to draw conclusions on how to build more sophisticated models.