Using Proxy Pattern-Mixture Models to Explain Bias in Estimates of COVID-19 Vaccine Uptake from Two Large Surveys

TL;DR

This paper applies proxy pattern-mixture models to large survey data to correct for bias in COVID-19 vaccine uptake estimates caused by non-ignorable nonresponse, providing more accurate and bounded estimates.

Contribution

It introduces the use of proxy pattern-mixture models for bias correction in large surveys with low response rates, specifically applied to COVID-19 vaccine data.

Findings

PPMM could detect bias direction in vaccine uptake estimates.

PPMM provided meaningful bias bounds for survey estimates.

Estimated vaccine hesitancy aligned with survey data but with adjusted bounds.

Abstract

Recently, attention was drawn to the failure of two very large internet-based probability surveys to correctly estimate COVID-19 vaccine uptake in the United States in early 2021. Both the Delphi-Facebook CTIS and Census Household Pulse Survey (HPS) overestimated uptake substantially, by 17 and 14 percentage points in May 2021, respectively. These surveys had large numbers of respondents but very low response rates (<10%), thus, non-ignorable nonresponse could have had substantial impact. Specifically, it is plausible that "anti-vaccine" individuals were less likely to participate given the topic (impact of the pandemic on daily life). In this paper we use proxy pattern-mixture models (PPMMs) to estimate the proportion of adults (18+) who received at least one dose of a COVID-19 vaccine, using data from the CTIS and HPS, under a non-ignorable nonresponse assumption. Data from the American Community Survey provide the necessary population data for the PPMMs. We compare these estimates to the true benchmark uptake numbers and show that the PPMM could have detected the direction of the bias and provide meaningful bias bounds. We also use the PPMM to estimate vaccine hesitancy, a measure for which we do not have a benchmark truth, and compare to the direct survey estimates.