Analyzing COVID-19 Tweets with Transformer-based Language Models

TL;DR

This paper presents a method using Transformer-based Language Models trained on COVID-19 tweets to analyze public opinion and biases, providing insights similar to polling on social and health issues.

Contribution

It introduces a novel approach of using prompt-based querying of TLMs trained on social media data to understand public opinion and biases.

Findings

Transformer models effectively reveal user biases.

Models produce polling-like insights on social and health topics.

Approach scales to large social media datasets.

Abstract

This paper describes a method for using Transformer-based Language Models (TLMs) to understand public opinion from social media posts. In this approach, we train a set of GPT models on several COVID-19 tweet corpora that reflect populations of users with distinctive views. We then use prompt-based queries to probe these models to reveal insights into the biases and opinions of the users. We demonstrate how this approach can be used to produce results which resemble polling the public on diverse social, political and public health issues. The results on the COVID-19 tweet data show that transformer language models are promising tools that can help us understand public opinions on social media at scale.