The COVID That Wasn't: Counterfactual Journalism Using GPT

TL;DR

This study uses GPT to generate COVID-19 news articles based on real headlines, revealing differences in tone and framing compared to actual news, and demonstrating a method to simulate cultural narratives.

Contribution

It introduces a novel approach using pre-2020 language models to generate and analyze news content, highlighting differences in tone and framing during the pandemic.

Findings

Generated articles are more negative about COVID

Artificial articles show less geopolitical framing

Method enables simulation of cultural narratives

Abstract

In this paper, we explore the use of large language models to assess human interpretations of real world events. To do so, we use a language model trained prior to 2020 to artificially generate news articles concerning COVID-19 given the headlines of actual articles written during the pandemic. We then compare stylistic qualities of our artificially generated corpus with a news corpus, in this case 5,082 articles produced by CBC News between January 23 and May 5, 2020. We find our artificially generated articles exhibits a considerably more negative attitude towards COVID and a significantly lower reliance on geopolitical framing. Our methods and results hold importance for researchers seeking to simulate large scale cultural processes via recent breakthroughs in text generation.