Should we tweet this? Generative response modeling for predicting reception of public health messaging on Twitter

TL;DR

This paper develops a generative response model using GPT-2 to predict and optimize public reactions to health messages on Twitter, aiding organizations in understanding and improving message reception during health crises.

Contribution

It introduces a novel predictive method leveraging GPT-2 for modeling Twitter responses to public health messages, with a new evaluation scheme demonstrating semantic and sentiment accuracy.

Findings

The model accurately captures the semantics of responses.

It can predict probable future reactions to health messages.

The evaluation scheme confirms the model's effectiveness.

Abstract

The way people respond to messaging from public health organizations on social media can provide insight into public perceptions on critical health issues, especially during a global crisis such as COVID-19. It could be valuable for high-impact organizations such as the US Centers for Disease Control and Prevention (CDC) or the World Health Organization (WHO) to understand how these perceptions impact reception of messaging on health policy recommendations. We collect two datasets of public health messages and their responses from Twitter relating to COVID-19 and Vaccines, and introduce a predictive method which can be used to explore the potential reception of such messages. Specifically, we harness a generative model (GPT-2) to directly predict probable future responses and demonstrate how it can be used to optimize expected reception of important health guidance. Finally, we introduce a novel evaluation scheme with extensive statistical testing which allows us to conclude that our models capture the semantics and sentiment found in actual public health responses.