The Effectiveness of Digital Interventions on COVID-19 Attitudes and Beliefs

TL;DR

This study evaluates the effectiveness of social media public health campaigns during COVID-19, analyzing 819 experiments across 174 organizations to measure their impact on beliefs and knowledge at scale.

Contribution

It provides the largest analysis to date of online COVID-19 public health interventions, quantifying their average influence on beliefs and vaccine knowledge.

Findings

Campaigns influence beliefs by about 1% at baseline

Cost per influenced person is approximately $3.41

Campaigns are especially effective at increasing vaccine knowledge

Abstract

During the course of the COVID-19 pandemic, a common strategy for public health organizations around the world has been to launch interventions via advertising campaigns on social media. Despite this ubiquity, little has been known about their average effectiveness. We conduct a large-scale program evaluation of campaigns from 174 public health organizations on Facebook and Instagram that collectively reached 2.1 billion individuals and cost around \$40 million. We report the results of 819 randomized experiments that measured the impact of these campaigns across standardized, survey-based outcomes. We find on average these campaigns are effective at influencing self-reported beliefs, shifting opinions close to 1% at baseline with a cost per influenced person of about \$3.41. There is further evidence that campaigns are especially effective at influencing users' knowledge of how to get vaccines. Our results represent, to the best of our knowledge, the largest set of online public health interventions analyzed to date.